Implementation Status#
The following tables summarize the features available in the various officialArrow libraries. All libraries currently follow version 1.0.0 of the Arrowformat, or later minor versions that are compatible with version 1.0.0. SeeFormat Versioning and Stability for details about versioning. Unless otherwisestated, the Python, R, Ruby and C/GLib libraries follow the C++ Arrow library.
Data Types#
Data type(primitive) | C++ | Java | Go | JS | C# | Rust | Julia | Swift | nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
Null | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Boolean | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Int8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
UInt8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Float16 | ✓ | ✓ (1) | ✓ | ✓ | ✓ (2) | ✓ | ✓ | ✓ | |
Float32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Decimal32 | ✓ | ✓ | ✓ | ✓ | |||||
Decimal64 | ✓ | ✓ | ✓ | ✓ | |||||
Decimal128 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Decimal256 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Date32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Time32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Timestamp | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Duration | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Interval | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
Fixed Size Binary | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Binary | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Large Binary | ✓ | ✓ | ✓ | ✓ | (4) | ✓ | ✓ | ✓ | |
Utf8 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Large Utf8 | ✓ | ✓ | ✓ | ✓ | (4) | ✓ | ✓ | ✓ | |
Binary View | ✓ | ✓ | ✓ | ✓ | ✓ | ||||
Utf8 View | ✓ | ✓ | ✓ | ✓ | ✓ |
Data type(nested) | C++ | Java | Go | JS | C# | Rust | Julia | Swift | nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
Fixed Size List | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
List | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Large List | ✓ | ✓ | ✓ | (4) | ✓ | ✓ | ✓ | ||
List View | ✓ | ✓ | ✓ | ✓ | |||||
Large List View | ✓ | ✓ | ✓ | ||||||
Struct | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Map | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Dense Union | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Sparse Union | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Data type(special) | C++ | Java | Go | JS | C# | Rust | Julia | Swift | nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
Dictionary | ✓ | ✓ (3) | ✓ | ✓ | ✓ | ✓ (3) | ✓ | ✓ | |
Extension | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||
Run-End Encoded | ✓ | ✓ | ✓ |
CanonicalExtension types | C++ | Java | Go | JavaScript | C# | Rust | Julia | Swift |
|---|---|---|---|---|---|---|---|---|
Fixed shape tensor | ✓ | |||||||
Variable shape tensor | ||||||||
JSON | ✓ | ✓ | ||||||
Opaque | ✓ | ✓ | ✓ | |||||
UUID | ✓ | ✓ | ||||||
8-bit Boolean | ✓ | ✓ | ||||||
Parquet Variant | ✓ |
Notes:
(1) Casting to/from Float16 in Java is not supported.
(2) Float16 support in C# is only available when targeting .NET 6+.
(3) Nested dictionaries not supported
(4) C# large array types are provided to help with interoperability with other libraries,but these do not support buffers larger than 2 GiB and an exception will be raised if trying to import an array that is too large.
See also
TheArrow Columnar Format and theCanonical Extension Types specification.
IPC Format#
IPC Feature | C++ | Java | Go | JS | C# | Rust | Julia | Swift | nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
Arrow stream format | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ (4) |
Arrow file format | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Record batches | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Dictionaries | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
Replacement dictionaries | ✓ | ✓ | ✓ | ✓ | |||||
Delta dictionaries | ✓ (1) | ✓ (1) | ✓ | ✓ | ✓ | ||||
Tensors | ✓ | ||||||||
Sparse tensors | ✓ | ||||||||
Buffer compression | ✓ | ✓ (3) | ✓ | ✓ | ✓ | ✓ | |||
Endianness conversion | ✓ (2) | ✓ (2) | ✓ (2) | ||||||
Custom schema metadata | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Notes:
(1) Delta dictionaries not supported on nested dictionaries
(2) Data with non-native endianness can be byte-swapped automatically when reading.
(3) LZ4 Codec currently is quite inefficient. ARROW-11901 tracks improving performance.
(4) The nanoarrow IPC implementation is only implemented for reading IPC streams.
See also
TheSerialization and Interprocess Communication (IPC) specification.
Flight RPC#
Flight RPC Transport | C++ | Java | Go | JS | C# | Rust | Julia | Swift |
|---|---|---|---|---|---|---|---|---|
gRPC transport (grpc:, grpc+tcp:) | ✓ | ✓ | ✓ | ✓ | ✓ | |||
gRPC domain socket transport (grpc+unix:) | ✓ | ✓ | ✓ | ✓ | ✓ | |||
gRPC + TLS transport (grpc+tls:) | ✓ | ✓ | ✓ | ✓ | ✓ |
Supported features in the gRPC transport:
Flight RPC Feature | C++ | Java | Go | JS | C# | Rust | Julia | Swift |
|---|---|---|---|---|---|---|---|---|
All RPC methods | ✓ | ✓ | ✓ | ✓ | ✓ | |||
Authentication handlers | ✓ | ✓ | ✓ | ✓ (1) | ✓ | |||
Call timeouts | ✓ | ✓ | ✓ | ✓ | ||||
Call cancellation | ✓ | ✓ | ✓ | ✓ | ||||
Concurrent client calls (2) | ✓ | ✓ | ✓ | ✓ | ✓ | |||
Custom middleware | ✓ | ✓ | ✓ | ✓ | ||||
RPC error codes | ✓ | ✓ | ✓ | ✓ | ✓ |
Notes:
(1) Support using AspNetCore authentication handlers.
(2) Whether a single client can support multiple concurrent calls.
See also
TheArrow Flight RPC specification.
Flight SQL#
Note
Flight SQL is still experimental.
The feature support refers to the client/server libraries only;databases which implement the Flight SQL protocol in turn willsupport/not support individual features.
Feature | C++ | Java | Go | JS | C# | Rust | Julia | Swift |
|---|---|---|---|---|---|---|---|---|
BeginSavepoint | ✓ | ✓ | ||||||
BeginTransaction | ✓ | ✓ | ||||||
CancelQuery | ✓ | ✓ | ||||||
ClosePreparedStatement | ✓ | ✓ | ✓ | ✓ | ✓ | |||
CreatePreparedStatement | ✓ | ✓ | ✓ | ✓ | ✓ | |||
CreatePreparedSubstraitPlan | ✓ | ✓ | ||||||
EndSavepoint | ✓ | ✓ | ||||||
EndTransaction | ✓ | ✓ | ||||||
GetCatalogs | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetCrossReference | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetDbSchemas | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetExportedKeys | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetImportedKeys | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetPrimaryKeys | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetSqlInfo | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetTables | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetTableTypes | ✓ | ✓ | ✓ | ✓ | ✓ | |||
GetXdbcTypeInfo | ✓ | ✓ | ✓ | ✓ | ✓ | |||
PreparedStatementQuery | ✓ | ✓ | ✓ | ✓ | ✓ | |||
PreparedStatementUpdate | ✓ | ✓ | ✓ | ✓ | ✓ | |||
StatementSubstraitPlan | ✓ | ✓ | ||||||
StatementQuery | ✓ | ✓ | ✓ | ✓ | ✓ | |||
StatementUpdate | ✓ | ✓ | ✓ | ✓ | ✓ |
See also
TheArrow Flight SQL specification.
C Data Interface#
Feature | C++ | Python | R | Rust | Go | Java | C/GLib | Ruby | Julia | C# | Swift | nanoarrow |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Schema export | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
Array export | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
Schema import | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
Array import | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
See also
TheC Data Interface specification.
C Stream Interface#
Feature | C++ | Python | R | Rust | Go | Java | C/GLib | Ruby | Julia | C# | Swift | nanoarrow |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Stream export | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||
Stream import | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
See also
TheC Stream Interface specification.
Third-Party Data Formats#
Format | C++ | Java | Go | JS | C# | Rust | Julia | Swift |
|---|---|---|---|---|---|---|---|---|
Avro | R | R | ||||||
CSV | R/W | R (2) | R/W | R/W | R/W | |||
ORC | R/W | R (1) | ||||||
Parquet | R/W | R (2) | R/W | R/W |
Notes:
R = Read supported
W = Write supported
(1) Through JNI bindings. (Provided by
org.apache.arrow.orc:arrow-orc)(2) Through JNI bindings to Arrow C++ Datasets. (Provided by
org.apache.arrow:arrow-dataset)