Reader classes: arrow-glib Reference Manual

Reader classes

Functions

GArrowSchema *	garrow_record_batch_reader_get_schema ()
GArrowRecordBatch *	garrow_record_batch_reader_get_next_record_batch ()
GArrowRecordBatch *	garrow_record_batch_reader_read_next_record_batch ()
GArrowRecordBatch *	garrow_record_batch_reader_read_next ()
GArrowTableBatchReader *	garrow_table_batch_reader_new ()
GArrowRecordBatchStreamReader *	garrow_record_batch_stream_reader_new ()
GArrowRecordBatchFileReader *	garrow_record_batch_file_reader_new ()
GArrowSchema *	garrow_record_batch_file_reader_get_schema ()
guint	garrow_record_batch_file_reader_get_n_record_batches ()
GArrowMetadataVersion	garrow_record_batch_file_reader_get_version ()
GArrowRecordBatch *	garrow_record_batch_file_reader_get_record_batch ()
GArrowRecordBatch *	garrow_record_batch_file_reader_read_record_batch ()
GArrowFeatherFileReader *	garrow_feather_file_reader_new ()
gchar *	garrow_feather_file_reader_get_description ()
gboolean	garrow_feather_file_reader_has_description ()
gint	garrow_feather_file_reader_get_version ()
gint64	garrow_feather_file_reader_get_n_rows ()
gint64	garrow_feather_file_reader_get_n_columns ()
gchar *	garrow_feather_file_reader_get_column_name ()
GArrowColumn *	garrow_feather_file_reader_get_column ()
GList *	garrow_feather_file_reader_get_columns ()
GArrowTable *	garrow_feather_file_reader_read ()
GArrowTable *	garrow_feather_file_reader_read_indices ()
GArrowTable *	garrow_feather_file_reader_read_names ()
GArrowCSVReadOptions *	garrow_csv_read_options_new ()
void	garrow_csv_read_options_add_column_type ()
void	garrow_csv_read_options_add_schema ()
GHashTable *	garrow_csv_read_options_get_column_types ()
GArrowCSVReader *	garrow_csv_reader_new ()
GArrowTable *	garrow_csv_reader_read ()

Properties

gpointer	record-batch-file-reader	Write / Construct Only
gboolean	allow-newlines-in-values	Read / Write
gint	block-size	Read / Write
gboolean	check-utf8	Read / Write
gchar	delimiter	Read / Write
gchar	escape-character	Read / Write
gboolean	ignore-empty-lines	Read / Write
gboolean	is-double-quoted	Read / Write
gboolean	is-escaped	Read / Write
gboolean	is-quoted	Read / Write
guint	n-header-rows	Read / Write
gpointer	pool	Write / Construct Only
gchar	quote-character	Read / Write
gboolean	use-threads	Read / Write
gpointer	csv-table-reader	Write / Construct Only
gpointer	feather-table-reader	Write / Construct Only
gpointer	record-batch-reader	Write / Construct Only

Types and Values

#define	GARROW_TYPE_RECORD_BATCH_READER
struct	GArrowRecordBatchReaderClass
#define	GARROW_TYPE_TABLE_BATCH_READER
struct	GArrowTableBatchReaderClass
struct	GArrowRecordBatchStreamReader
struct	GArrowRecordBatchFileReader
#define	GARROW_TYPE_FEATHER_FILE_READER
struct	GArrowFeatherFileReaderClass
#define	GARROW_TYPE_CSV_READ_OPTIONS
struct	GArrowCSVReadOptionsClass
#define	GARROW_TYPE_CSV_READER
struct	GArrowCSVReaderClass
	GArrowCSVReadOptions
	GArrowCSVReader
	GArrowFeatherFileReader
	GArrowRecordBatchReader
	GArrowTableBatchReader

Object Hierarchy

    GObject
    ├── GArrowCSVReadOptions
    ├── GArrowCSVReader
    ├── GArrowFeatherFileReader
    ├── GArrowRecordBatchFileReader
    ╰── GArrowRecordBatchReader
        ├── GArrowRecordBatchStreamReader
        ├── GArrowTableBatchReader
        ╰── GArrowRecordBatchStreamReader

Includes

#include <arrow-glib/arrow-glib.h>

Description

GArrowRecordBatchReader is a base class for reading record batches in stream format from input.

GArrowRecordBatchStreamReader is a class for reading record batches in stream format from input synchronously.

GArrowRecordBatchFileReader is a class for reading record batches in file format from input.

GArrowFeatherFileReader is a class for reading columns in Feather file format from input.

GArrowCSVReader is a class for reading table in CSV format from input.

Functions

garrow_record_batch_reader_get_schema ()

GArrowSchema *
garrow_record_batch_reader_get_schema (GArrowRecordBatchReader *reader);

Parameters

reader

A GArrowRecordBatchReader.

Returns

The schema in the stream.

[transfer full]

Since: 0.4.0

garrow_record_batch_reader_get_next_record_batch ()

GArrowRecordBatch *
garrow_record_batch_reader_get_next_record_batch
                               (GArrowRecordBatchReader *reader,
                                GError **error);

garrow_record_batch_reader_get_next_record_batch has been deprecated since version 0.5.0 and should not be used in newly-written code.

Use garrow_record_batch_reader_read_next() instead.

Parameters

reader	A GArrowRecordBatchReader.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The next record batch in the stream or NULL on end of stream.

[nullable][transfer full]

Since: 0.4.0

garrow_record_batch_reader_read_next_record_batch ()

GArrowRecordBatch *
garrow_record_batch_reader_read_next_record_batch
                               (GArrowRecordBatchReader *reader,
                                GError **error);

garrow_record_batch_reader_read_next_record_batch has been deprecated since version 0.8.0 and should not be used in newly-written code.

Use garrow_record_batch_reader_read_next() instead.

Parameters

reader	A GArrowRecordBatchReader.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The next record batch in the stream or NULL on end of stream.

[nullable][transfer full]

Since: 0.5.0

garrow_record_batch_reader_read_next ()

GArrowRecordBatch *
garrow_record_batch_reader_read_next (GArrowRecordBatchReader *reader,
                                      GError **error);

Parameters

reader	A GArrowRecordBatchReader.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The next record batch in the stream or NULL on end of stream.

[nullable][transfer full]

Since: 0.8.0

garrow_table_batch_reader_new ()

GArrowTableBatchReader *
garrow_table_batch_reader_new (GArrowTable *table);

Parameters

table

The table to be read.

Returns

A newly created GArrowTableBatchReader.

Since: 0.8.0

garrow_record_batch_stream_reader_new ()

GArrowRecordBatchStreamReader *
garrow_record_batch_stream_reader_new (GArrowInputStream *stream,
                                       GError **error);

Parameters

stream	The stream to be read.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowRecordBatchStreamReader or NULL on error.

[nullable]

Since: 0.4.0

garrow_record_batch_file_reader_new ()

GArrowRecordBatchFileReader *
garrow_record_batch_file_reader_new (GArrowSeekableInputStream *file,
                                     GError **error);

Parameters

file	The file to be read.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowRecordBatchFileReader or NULL on error.

[nullable]

Since: 0.4.0

garrow_record_batch_file_reader_get_schema ()

GArrowSchema *
garrow_record_batch_file_reader_get_schema
                               (GArrowRecordBatchFileReader *reader);

Parameters

reader

A GArrowRecordBatchFileReader.

Returns

The schema in the file.

[transfer full]

Since: 0.4.0

garrow_record_batch_file_reader_get_n_record_batches ()

guint
garrow_record_batch_file_reader_get_n_record_batches
                               (GArrowRecordBatchFileReader *reader);

Parameters

reader

A GArrowRecordBatchFileReader.

Returns

The number of record batches in the file.

Since: 0.4.0

garrow_record_batch_file_reader_get_version ()

GArrowMetadataVersion
garrow_record_batch_file_reader_get_version
                               (GArrowRecordBatchFileReader *reader);

Parameters

reader

A GArrowRecordBatchFileReader.

Returns

The format version in the file.

Since: 0.4.0

garrow_record_batch_file_reader_get_record_batch ()

GArrowRecordBatch *
garrow_record_batch_file_reader_get_record_batch
                               (GArrowRecordBatchFileReader *reader,
                                guint i,
                                GError **error);

garrow_record_batch_file_reader_get_record_batch has been deprecated since version 0.5.0 and should not be used in newly-written code.

Use garrow_record_batch_file_reader_read_record_batch() instead.

Parameters

reader	A GArrowRecordBatchFileReader.
i	The index of the target record batch.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The i-th record batch in the file or NULL on error.

[nullable][transfer full]

Since: 0.4.0

garrow_record_batch_file_reader_read_record_batch ()

GArrowRecordBatch *
garrow_record_batch_file_reader_read_record_batch
                               (GArrowRecordBatchFileReader *reader,
                                guint i,
                                GError **error);

Parameters

reader	A GArrowRecordBatchFileReader.
i	The index of the target record batch.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The i-th record batch in the file or NULL on error.

[nullable][transfer full]

Since: 0.5.0

garrow_feather_file_reader_new ()

GArrowFeatherFileReader *
garrow_feather_file_reader_new (GArrowSeekableInputStream *file,
                                GError **error);

Parameters

file	The file to be read.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowFeatherFileReader or NULL on error.

[nullable]

Since: 0.4.0

garrow_feather_file_reader_get_description ()

gchar *
garrow_feather_file_reader_get_description
                               (GArrowFeatherFileReader *reader);

Parameters

reader

A GArrowFeatherFileReader.

Returns

The description of the file if it exists, NULL otherwise. You can confirm whether description exists or not by garrow_feather_file_reader_has_description().

It should be freed with g_free() when no longer needed.

[nullable][transfer full]

Since: 0.4.0

garrow_feather_file_reader_has_description ()

gboolean
garrow_feather_file_reader_has_description
                               (GArrowFeatherFileReader *reader);

Parameters

reader

A GArrowFeatherFileReader.

Returns

Whether the file has description or not.

Since: 0.4.0

garrow_feather_file_reader_get_version ()

gint
garrow_feather_file_reader_get_version
                               (GArrowFeatherFileReader *reader);

Parameters

reader

A GArrowFeatherFileReader.

Returns

The format version of the file.

Since: 0.4.0

garrow_feather_file_reader_get_n_rows ()

gint64
garrow_feather_file_reader_get_n_rows (GArrowFeatherFileReader *reader);

Parameters

reader

A GArrowFeatherFileReader.

Returns

The number of rows in the file.

Since: 0.4.0

garrow_feather_file_reader_get_n_columns ()

gint64
garrow_feather_file_reader_get_n_columns
                               (GArrowFeatherFileReader *reader);

Parameters

reader

A GArrowFeatherFileReader.

Returns

The number of columns in the file.

Since: 0.4.0

garrow_feather_file_reader_get_column_name ()

gchar *
garrow_feather_file_reader_get_column_name
                               (GArrowFeatherFileReader *reader,
                                gint i);

Parameters

reader	A GArrowFeatherFileReader.
i	The index of the target column.

Returns

The i-th column name in the file.

It should be freed with g_free() when no longer needed.

[transfer full]

Since: 0.4.0

garrow_feather_file_reader_get_column ()

GArrowColumn *
garrow_feather_file_reader_get_column (GArrowFeatherFileReader *reader,
                                       gint i,
                                       GError **error);

Parameters

reader	A GArrowFeatherFileReader.
i	The index of the target column.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The i-th column in the file or NULL on error.

[nullable][transfer full]

Since: 0.4.0

garrow_feather_file_reader_get_columns ()

GList *
garrow_feather_file_reader_get_columns
                               (GArrowFeatherFileReader *reader,
                                GError **error);

Parameters

reader	A GArrowFeatherFileReader.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The columns in the file.

[element-type GArrowColumn][transfer full]

Since: 0.4.0

garrow_feather_file_reader_read ()

GArrowTable *
garrow_feather_file_reader_read (GArrowFeatherFileReader *reader,
                                 GError **error);

Parameters

reader	A GArrowFeatherFileReader.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The table in the file that has all columns.

[transfer full]

Since: 0.12.0

garrow_feather_file_reader_read_indices ()

GArrowTable *
garrow_feather_file_reader_read_indices
                               (GArrowFeatherFileReader *reader,
                                const gint *indices,
                                guint n_indices,
                                GError **error);

Parameters

reader	A GArrowFeatherFileReader.
indices	The indices of column to be read.	[array length=n_indices]
n_indices	The number of indices.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The table in the file that has only the specified columns.

[transfer full]

Since: 0.12.0

garrow_feather_file_reader_read_names ()

GArrowTable *
garrow_feather_file_reader_read_names (GArrowFeatherFileReader *reader,
                                       const gchar **names,
                                       guint n_names,
                                       GError **error);

Parameters

reader	A GArrowFeatherFileReader.
names	The names of column to be read.	[array length=n_names]
n_names	The number of names.
error	Return location for a GError or `NULL`.	[nullable]

Returns

The table in the file that has only the specified columns.

[transfer full]

Since: 0.12.0

garrow_csv_read_options_new ()

GArrowCSVReadOptions *
garrow_csv_read_options_new (void);

Returns

A newly created GArrowCSVReadOptions.

Since: 0.12.0

garrow_csv_read_options_add_column_type ()

void
garrow_csv_read_options_add_column_type
                               (GArrowCSVReadOptions *options,
                                const gchar *name,
                                GArrowDataType *data_type);

Add value type of a column.

Parameters

options	A GArrowCSVReadOptions.
name	The name of the target column.
data_type	The GArrowDataType for the column.

Since: 0.12.0

garrow_csv_read_options_add_schema ()

void
garrow_csv_read_options_add_schema (GArrowCSVReadOptions *options,
                                    GArrowSchema *schema);

Add value types for columns in the schema.

Parameters

options	A GArrowCSVReadOptions.
schema	The GArrowSchema that specifies columns and their types.

Since: 0.12.0

garrow_csv_read_options_get_column_types ()

GHashTable *
garrow_csv_read_options_get_column_types
                               (GArrowCSVReadOptions *options);

Parameters

options

A GArrowCSVReadOptions.

Returns

The column name and value type mapping of the options.

[transfer full][element-type gchar* GArrowDataType]

Since: 0.12.0

garrow_csv_reader_new ()

GArrowCSVReader *
garrow_csv_reader_new (GArrowInputStream *input,
                       GArrowCSVReadOptions *options,
                       GError **error);

Parameters

input	The input to be read.
options	A GArrowCSVReadOptions.	[nullable]
error	Return location for a GError or `NULL`.	[nullable]

Returns

A newly created GArrowCSVReader or NULL on error.

[nullable]

Since: 0.12.0

garrow_csv_reader_read ()

GArrowTable *
garrow_csv_reader_read (GArrowCSVReader *reader,
                        GError **error);

Parameters

reader	A GArrowCSVReader.
error	Return location for a GError or `NULL`.	[nullable]

Returns

A read GArrowTable or NULL on error.

[nullable][transfer full]

Since: 0.12.0

Types and Values

GARROW_TYPE_RECORD_BATCH_READER

#define GARROW_TYPE_RECORD_BATCH_READER (garrow_record_batch_reader_get_type())

struct GArrowRecordBatchReaderClass

struct GArrowRecordBatchReaderClass {
  GObjectClass parent_class;
};

GARROW_TYPE_TABLE_BATCH_READER

#define GARROW_TYPE_TABLE_BATCH_READER (garrow_table_batch_reader_get_type())

struct GArrowTableBatchReaderClass

struct GArrowTableBatchReaderClass {
  GArrowRecordBatchReaderClass parent_class;
};

struct GArrowRecordBatchStreamReader

struct GArrowRecordBatchStreamReader;

It wraps arrow::ipc::RecordBatchStreamReader.

struct GArrowRecordBatchFileReader

struct GArrowRecordBatchFileReader;

It wraps arrow::ipc::RecordBatchFileReader.

GARROW_TYPE_FEATHER_FILE_READER

#define GARROW_TYPE_FEATHER_FILE_READER (garrow_feather_file_reader_get_type())

struct GArrowFeatherFileReaderClass

struct GArrowFeatherFileReaderClass {
  GObjectClass parent_class;
};

GARROW_TYPE_CSV_READ_OPTIONS

#define GARROW_TYPE_CSV_READ_OPTIONS (garrow_csv_read_options_get_type())

struct GArrowCSVReadOptionsClass

struct GArrowCSVReadOptionsClass {
  GObjectClass parent_class;
};

GARROW_TYPE_CSV_READER

#define GARROW_TYPE_CSV_READER (garrow_csv_reader_get_type())

struct GArrowCSVReaderClass

struct GArrowCSVReaderClass {
  GObjectClass parent_class;
};

GArrowCSVReadOptions

typedef struct _GArrowCSVReadOptions GArrowCSVReadOptions;

GArrowCSVReader

typedef struct _GArrowCSVReader GArrowCSVReader;

GArrowFeatherFileReader

typedef struct _GArrowFeatherFileReader GArrowFeatherFileReader;

GArrowRecordBatchReader

typedef struct _GArrowRecordBatchReader GArrowRecordBatchReader;

GArrowTableBatchReader

typedef struct _GArrowTableBatchReader GArrowTableBatchReader;

Property Details

The `“record-batch-file-reader”` property

  “record-batch-file-reader” gpointer

The raw std::shared<arrow::ipc::RecordBatchFileReader> *.

Flags: Write / Construct Only

The `“allow-newlines-in-values”` property

  “allow-newlines-in-values” gboolean

Whether values are allowed to contain CR (0x0d) and LF (0x0a) characters.

Flags: Read / Write

Default value: FALSE

Since: 0.12.0

The `“block-size”` property

  “block-size”               gint

Block size we request from the IO layer; also determines the size of chunks when “use-threads” is TRUE.

Flags: Read / Write

Allowed values: >= 0

Default value: 1048576

Since: 0.12.0

The `“check-utf8”` property

  “check-utf8”               gboolean

Whether to check UTF8 validity of string columns.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0

The `“delimiter”` property

  “delimiter”                gchar

Field delimiter character.

Flags: Read / Write

Allowed values: >= 0

Default value: 44

Since: 0.12.0

The `“escape-character”` property

  “escape-character”         gchar

Escaping character. This is used only when “is-escaped” is TRUE.

Flags: Read / Write

Allowed values: >= 0

Default value: 92

Since: 0.12.0

The `“ignore-empty-lines”` property

  “ignore-empty-lines”       gboolean

Whether empty lines are ignored. If FALSE, an empty line represents a simple empty value (assuming a one-column CSV file).

Flags: Read / Write

Default value: TRUE

Since: 0.12.0

The `“is-double-quoted”` property

  “is-double-quoted”         gboolean

Whether a quote inside a value is double quoted.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0

The `“is-escaped”` property

  “is-escaped”               gboolean

Whether escaping is used.

Flags: Read / Write

Default value: FALSE

Since: 0.12.0

The `“is-quoted”` property

  “is-quoted”                gboolean

Whether quoting is used.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0

The `“n-header-rows”` property

  “n-header-rows”            guint

The number of header rows to skip (including the first row containing column names)

Flags: Read / Write

Default value: 1

Since: 0.12.0

The `“pool”` property

  “pool”                     gpointer

The raw arrow::MemoryPool *.

Flags: Write / Construct Only

The `“quote-character”` property

  “quote-character”          gchar

Quoting character. This is used only when “is-quoted” is TRUE.

Flags: Read / Write

Allowed values: >= 0

Default value: 34

Since: 0.12.0

The `“use-threads”` property

  “use-threads”              gboolean

Whether to use the global CPU thread pool.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0

The `“csv-table-reader”` property

  “csv-table-reader”         gpointer

The raw std::shared<arrow::csv::TableReader> *.

Flags: Write / Construct Only

The `“feather-table-reader”` property

  “feather-table-reader”     gpointer

The raw std::shared<arrow::ipc::feather::TableReader> *.

Flags: Write / Construct Only

The `“record-batch-reader”` property

  “record-batch-reader”      gpointer

The raw std::shared<arrow::ipc::RecordBatchRecordBatchReader> *.

Flags: Write / Construct Only