Array Builders#
- classArrayBuilder#
Base class for all data array builders.
This class provides a facilities for incrementally building the null bitmap (see Append methods) and as a side effect the current number of slots and the null count.
Note
Users are expected to use builders as one of the concrete types below. For example, ArrayBuilder* pointing toBinaryBuilder should be downcast before use.
Subclassed byarrow::BaseBinaryBuilder< BinaryType >,arrow::BaseBinaryBuilder< LargeBinaryType >,arrow::NumericBuilder< DayTimeIntervalType >,arrow::NumericBuilder< HalfFloatType >,arrow::NumericBuilder< MonthDayNanoIntervalType >,arrow::VarLengthListLikeBuilder< LargeListType >,arrow::VarLengthListLikeBuilder< LargeListViewType >,arrow::VarLengthListLikeBuilder< ListType >,arrow::VarLengthListLikeBuilder< ListViewType >, arrow::internal::DictionaryBuilderBase< Int32Builder, T >, arrow::internal::DictionaryBuilderBase< AdaptiveIntBuilder, T >,arrow::BaseBinaryBuilder< TYPE >,arrow::BasicUnionBuilder,arrow::BinaryViewBuilder,arrow::BooleanBuilder,arrow::FixedSizeBinaryBuilder,arrow::FixedSizeListBuilder,arrow::MapBuilder,arrow::NullBuilder,arrow::NumericBuilder< T >, arrow::RunEndEncodedBuilder,arrow::StructBuilder,arrow::VarLengthListLikeBuilder< TYPE >, arrow::internal::AdaptiveIntBuilderBase, arrow::internal::DictionaryBuilderBase< BuilderType, T >, arrow::internal::DictionaryBuilderBase< BuilderType, NullType >, arrow::internal::RunCompressorBuilder
Public Functions
- inlineArrayBuilder*child(inti)#
For nested types.
Since the objects are owned by this class instance, we skip shared pointers and just return a raw pointer
- virtualStatusResize(int64_tcapacity)#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlineStatusReserve(int64_tadditional_capacity)#
Ensure that there is enough space allocated to append the indicated number of elements without any further reallocation.
Overallocation is used in order to minimize the impact of incrementalReserve() calls. Note that additional_capacity is relative to the current number of elements rather than to the current capacity, so calls toReserve() which are not interspersed with addition of new elements may not increase the capacity.
- Parameters:
additional_capacity –[in] the number of additional array values
- Returns:
- virtualvoidReset()#
Reset the builder.
- virtualStatusAppendEmptyValue()=0#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- virtualStatusAppendEmptyValues(int64_tlength)=0#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)#
Append a range of values from an array.
The given array must be the same type as the builder.
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)=0#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- StatusFinish(std::shared_ptr<Array>*out)#
Return result of builder as anArray object.
The builder is reset except forDictionaryBuilder.
- inlineArrayBuilder*child(inti)#
Concrete builder subclasses#
Primitive#
- classNullBuilder:publicarrow::ArrayBuilder#
Public Functions
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlinevirtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- inlinevirtualStatusAppendArraySlice(constArraySpan&,int64_t,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
- classBooleanBuilder:publicarrow::ArrayBuilder,publicarrow::internal::ArrayBuilderExtraOps<BooleanBuilder,bool>#
Public Functions
- inlinevirtualStatusAppendNulls(int64_tlength)final#
Write nulls as uint8_t* (0 value indicates null) into pre-allocated memory.
- inlinevirtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- StatusAppendValues(constuint8_t*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous array of bytes (non-zero is 1)
length –[in] the number of values to append
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- StatusAppendValues(constuint8_t*values,int64_tlength,constuint8_t*validity,int64_toffset)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a bitmap of values
length –[in] the number of values to append
validity –[in] a validity bitmap to copy (may be null)
offset –[in] an offset into the values and validity bitmaps
- Returns:
- StatusAppendValues(constuint8_t*values,int64_tlength,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous C array of values
length –[in] the number of values to append
is_valid –[in] an std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- StatusAppendValues(conststd::vector<uint8_t>&values,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a std::vector of bytes
is_valid –[in] an std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- StatusAppendValues(conststd::vector<uint8_t>&values)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a std::vector of bytes
- Returns:
- StatusAppendValues(conststd::vector<bool>&values,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] an std::vector<bool> indicating true (1) or false
is_valid –[in] an std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- StatusAppendValues(conststd::vector<bool>&values)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] an std::vector<bool> indicating true (1) or false
- Returns:
- template<typenameValuesIter>
inlineStatusAppendValues(ValuesItervalues_begin,ValuesItervalues_end)# Append a sequence of elements in one shot.
- Parameters:
values_begin –[in] InputIterator to the beginning of the values
values_end –[in] InputIterator pointing to the end of the values or null(0) values
- Returns:
- template<typenameValuesIter,typenameValidIter>
inlineenable_if_t<!std::is_pointer<ValidIter>::value,Status>AppendValues(ValuesItervalues_begin,ValuesItervalues_end,ValidItervalid_begin)# Append a sequence of elements in one shot, with a specified nullmap.
- Parameters:
values_begin –[in] InputIterator to the beginning of the values
values_end –[in] InputIterator pointing to the end of the values
valid_begin –[in] InputIterator with elements indication valid(1) or null(0) values
- Returns:
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- virtualvoidReset()override#
Reset the builder.
- virtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlinevirtualStatusAppendNulls(int64_tlength)final#
- usingDecimalBuilder=Decimal128Builder#
- usingUInt8Builder=NumericBuilder<UInt8Type>#
- usingUInt16Builder=NumericBuilder<UInt16Type>#
- usingUInt32Builder=NumericBuilder<UInt32Type>#
- usingUInt64Builder=NumericBuilder<UInt64Type>#
- usingInt8Builder=NumericBuilder<Int8Type>#
- usingInt16Builder=NumericBuilder<Int16Type>#
- usingInt32Builder=NumericBuilder<Int32Type>#
- usingInt64Builder=NumericBuilder<Int64Type>#
- usingFloatBuilder=NumericBuilder<FloatType>#
- usingDoubleBuilder=NumericBuilder<DoubleType>#
- classAdaptiveUIntBuilder:publicarrow::internal::AdaptiveIntBuilderBase#
- #include <arrow/array/builder_adaptive.h>
Public Functions
- StatusAppendValues(constuint64_t*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous C array of values
length –[in] the number of values to append
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- StatusAppendValues(constuint64_t*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
- classAdaptiveIntBuilder:publicarrow::internal::AdaptiveIntBuilderBase#
- #include <arrow/array/builder_adaptive.h>
Public Functions
- StatusAppendValues(constint64_t*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous C array of values
length –[in] the number of values to append
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- StatusAppendValues(constint64_t*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
- classDecimal32Builder:publicarrow::FixedSizeBinaryBuilder#
- #include <arrow/array/builder_decimal.h>
- classDecimal64Builder:publicarrow::FixedSizeBinaryBuilder#
- #include <arrow/array/builder_decimal.h>
- classDecimal128Builder:publicarrow::FixedSizeBinaryBuilder#
- #include <arrow/array/builder_decimal.h>
- classDecimal256Builder:publicarrow::FixedSizeBinaryBuilder#
- #include <arrow/array/builder_decimal.h>
- template<typenameT>
classNumericBuilder:publicarrow::ArrayBuilder,publicarrow::internal::ArrayBuilderExtraOps<NumericBuilder<T>,T::c_type># - #include <arrow/array/builder_primitive.h>
Base class for all Builders that emit anArray of a scalar numerical type.
Public Functions
- inlinevirtualStatusAppendNulls(int64_tlength)final#
Write nulls as uint8_t* (0 value indicates null) into pre-allocated memory The memory at the corresponding data slot is set to 0 to prevent uninitialized memory access.
- inlinevirtualvoidReset()override#
Reset the builder.
- inlinevirtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlineStatusAppendValues(constvalue_type*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous C array of values
length –[in] the number of values to append
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- inlineStatusAppendValues(constvalue_type*values,int64_tlength,constuint8_t*bitmap,int64_tbitmap_offset)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous C array of values
length –[in] the number of values to append
bitmap –[in] a validity bitmap to copy (may be null)
bitmap_offset –[in] an offset into the validity bitmap
- Returns:
- inlineStatusAppendValues(constvalue_type*values,int64_tlength,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous C array of values
length –[in] the number of values to append
is_valid –[in] an std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- inlineStatusAppendValues(conststd::vector<value_type>&values,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a std::vector of values
is_valid –[in] an std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- inlineStatusAppendValues(conststd::vector<value_type>&values)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a std::vector of values
- Returns:
- inlinevirtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- template<typenameValuesIter>
inlineStatusAppendValues(ValuesItervalues_begin,ValuesItervalues_end)# Append a sequence of elements in one shot.
- Parameters:
values_begin –[in] InputIterator to the beginning of the values
values_end –[in] InputIterator pointing to the end of the values
- Returns:
- template<typenameValuesIter,typenameValidIter>
inlineenable_if_t<!std::is_pointer<ValidIter>::value,Status>AppendValues(ValuesItervalues_begin,ValuesItervalues_end,ValidItervalid_begin)# Append a sequence of elements in one shot, with a specified nullmap.
- Parameters:
values_begin –[in] InputIterator to the beginning of the values
values_end –[in] InputIterator pointing to the end of the values
valid_begin –[in] InputIterator with elements indication valid(1) or null(0) values.
- Returns:
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- inlinevoidUnsafeAppend(constvalue_typeval)#
Append a single scalar under the assumption that the underlyingBuffer is large enough.
This method does not capacity-check; make sure to call Reserve beforehand.
- inlinevoidUnsafeAdvance(int64_tlength)#
Advance builder without allocating nor writing any values.
The internal pointer is advanced by
lengthvalues and the same number of non-null entries are appended to the validity bitmap. This method assumes that thelengthvalues were populated directly, for example usingGetMutableValue.
- inlinevoidUnsafeAdvance(int64_tlength,constuint8_t*validity,int64_tvalid_bits_offset)#
Advance builder without allocating nor writing any values.
The internal pointer is advanced by
lengthvalues and the same number of validity bits are appended to the validity bitmap. This method assumes that thelengthvalues were populated directly, for example usingGetMutableValue.
- inlinevirtualStatusAppendNulls(int64_tlength)final#
- classHalfFloatBuilder:publicarrow::NumericBuilder<HalfFloatType>#
- #include <arrow/array/builder_primitive.h>
Public Functions
- inlinevoidUnsafeAppend(constFloat16val)#
Scalar append a arrow::util::Float16, without checking for capacity.
- inlineStatusAppendValues(constFloat16*values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous array of arrow::util::Float16
length –[in] the number of values to append
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- inlineStatusAppendValues(constFloat16*values,int64_tlength,constuint8_t*bitmap,int64_tbitmap_offset)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous array of arrow::util::Float16
length –[in] the number of values to append
bitmap –[in] a validity bitmap to copy (may be null)
bitmap_offset –[in] an offset into the validity bitmap
- Returns:
- inlineStatusAppendValues(constFloat16*values,int64_tlength,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a contiguous array of arrow::util::Float16
length –[in] the number of values to append
is_valid –[in] a std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- inlineStatusAppendValues(conststd::vector<Float16>&values,conststd::vector<bool>&is_valid)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a std::vector<arrow::util::Float16>
is_valid –[in] a std::vector<bool> indicating valid (1) or null (0). Equal in length to values
- Returns:
- inlineStatusAppendValues(conststd::vector<Float16>&values)#
Append a sequence of elements in one shot.
- Parameters:
values –[in] a std::vector<arrow::util::Float16>
- Returns:
- inlinevoidUnsafeAppend(constFloat16val)#
Temporal#
- typedefNumericBuilder<Date32Type>Date32Builder#
- typedefNumericBuilder<Date64Type>Date64Builder#
- typedefNumericBuilder<Time32Type>Time32Builder#
- typedefNumericBuilder<Time64Type>Time64Builder#
- typedefNumericBuilder<TimestampType>TimestampBuilder#
- typedefNumericBuilder<MonthIntervalType>MonthIntervalBuilder#
- typedefNumericBuilder<DurationType>DurationBuilder#
- classDayTimeIntervalBuilder:publicarrow::NumericBuilder<DayTimeIntervalType>#
- #include <arrow/array/builder_time.h>
- classMonthDayNanoIntervalBuilder:publicarrow::NumericBuilder<MonthDayNanoIntervalType>#
- #include <arrow/array/builder_time.h>
Binary-like#
- template<typenameTYPE>
classBaseBinaryBuilder:publicarrow::ArrayBuilder,publicarrow::internal::ArrayBuilderExtraOps<BaseBinaryBuilder<TYPE>,std::string_view># - #include <arrow/array/builder_binary.h>
Public Functions
- inlineStatusExtendCurrent(constuint8_t*value,offset_typelength)#
Extend the last appended value by appending more data at the end.
Unlike Append, this does not create a new offset.
- inlinevirtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlinevoidUnsafeAppend(constuint8_t*value,offset_typelength)#
Append without checking capacity.
Offsets and data should have been presized usingReserve() andReserveData(), respectively.
- inlinevoidUnsafeExtendCurrent(constuint8_t*value,offset_typelength)#
Like ExtendCurrent, but do not check capacity.
- inlineStatusAppendValues(conststd::vector<std::string>&values,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of strings in one shot.
- Parameters:
values –[in] a vector of strings
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- inlineStatusAppendValues(constchar**values,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Append a sequence of nul-terminated strings in one shot.
If one of the values is NULL, it is processed as a null value even if the corresponding valid_bytes entry is 1.
- Parameters:
values –[in] a contiguous C array of nul-terminated char *
length –[in] the number of values to append
valid_bytes –[in] an optional sequence of bytes where non-zero indicates a valid (non-null) value
- Returns:
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- inlinevirtualvoidReset()override#
Reset the builder.
- inlinevirtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlineStatusReserveData(int64_telements)#
Ensures there is enough allocated capacity to append the indicated number of bytes to the value data buffer without additional allocations.
- inlinevirtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- inlineconstuint8_t*value_data()const#
- Returns:
data pointer of the value date builder
- inlineint64_tvalue_data_length()const#
- Returns:
size of values buffer so far
- inlineint64_tvalue_data_capacity()const#
- Returns:
capacity of values buffer
- inlineconstoffset_type*offsets_data()const#
- Returns:
data pointer of the value date builder
- inlineconstuint8_t*GetValue(int64_ti,offset_type*out_length)const#
Temporary access to a value.
This pointer becomes invalid on the next modifying operation.
- inlinestd::string_viewGetView(int64_ti)const#
Temporary access to a value.
This view becomes invalid on the next modifying operation.
- inlineStatusExtendCurrent(constuint8_t*value,offset_typelength)#
- classBinaryBuilder:publicarrow::BaseBinaryBuilder<BinaryType>#
- #include <arrow/array/builder_binary.h>
Builder class for variable-length binary data.
Subclassed byarrow::StringBuilder
- classStringBuilder:publicarrow::BinaryBuilder#
- #include <arrow/array/builder_binary.h>
Builder class for UTF8 strings.
- classLargeBinaryBuilder:publicarrow::BaseBinaryBuilder<LargeBinaryType>#
- #include <arrow/array/builder_binary.h>
Builder class for large variable-length binary data.
Subclassed byarrow::LargeStringBuilder
- classLargeStringBuilder:publicarrow::LargeBinaryBuilder#
- #include <arrow/array/builder_binary.h>
Builder class for large UTF8 strings.
- classBinaryViewBuilder:publicarrow::ArrayBuilder#
- #include <arrow/array/builder_binary.h>
Subclassed byarrow::StringViewBuilder
Public Functions
- inlinevoidSetBlockSize(int64_tblocksize)#
Set the size for future preallocated data buffers.
The default size is 32KB, so after each 32KB of string data appended to the builder a new data buffer will be allocated. Adjust this to a larger value to decrease the frequency of allocation, or to a smaller value to lower the overhead of each allocation.
- inlineint64_tcurrent_block_bytes_remaining()const#
The number of bytes which can be appended to this builder without allocating another data buffer.
- inlinevoidUnsafeAppend(constuint8_t*value,int64_tlength)#
Append without checking capacity.
Builder should have been presized usingReserve() andReserveData(), respectively, and the value must not be larger than 2GB
- StatusReserveData(int64_tlength)#
Ensures there is enough allocated available capacity in the out-of-line data heap to append the indicated number of bytes without additional allocations.
- virtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a slice of aBinaryViewArray passed as an ArraySpan.
Copies the underlying out-of-line string memory to avoid memory lifetime issues
- virtualvoidReset()override#
Reset the builder.
- inlinevirtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlinevoidSetBlockSize(int64_tblocksize)#
- classStringViewBuilder:publicarrow::BinaryViewBuilder#
- #include <arrow/array/builder_binary.h>
- classFixedSizeBinaryBuilder:publicarrow::ArrayBuilder#
- #include <arrow/array/builder_binary.h>
Subclassed byarrow::Decimal128Builder,arrow::Decimal256Builder,arrow::Decimal32Builder,arrow::Decimal64Builder
Public Functions
- virtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- virtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- inlineStatusReserveData(int64_telements)#
Ensures there is enough allocated capacity to append the indicated number of bytes to the value data buffer without additional allocations.
- virtualvoidReset()override#
Reset the builder.
- virtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- inlineint64_tvalue_data_length()const#
- Returns:
size of values buffer so far
- constuint8_t*GetValue(int64_ti)const#
Temporary access to a value.
This pointer becomes invalid on the next modifying operation.
- inlineuint8_t*GetMutableValue(int64_ti)#
Temporary mutable access to a value.
This pointer becomes invalid on the next modifying operation.
- std::string_viewGetView(int64_ti)const#
Temporary mutable access to a value.
This view becomes invalid on the next modifying operation.
- inlinevoidUnsafeAdvance(int64_tlength)#
Advance builder without allocating nor writing any values.
The internal pointer is advanced by
lengthvalues and the same number of non-null entries are appended to the validity bitmap. This method assumes that thelengthvalues were populated directly, for example usingGetMutableValue.
- inlinevoidUnsafeAdvance(int64_tlength,constuint8_t*validity,int64_tvalid_bits_offset)#
Advance builder without allocating nor writing any values.
The internal pointer is advanced by
lengthvalues and the same number of validity bits are appended to the validity bitmap. This method assumes that thelengthvalues were populated directly, for example usingGetMutableValue.
- virtualStatusAppendEmptyValue()final#
Nested#
- template<typenameTYPE>
classVarLengthListLikeBuilder:publicarrow::ArrayBuilder# - #include <arrow/array/builder_nested.h>
Subclassed byarrow::BaseListBuilder< TYPE >,arrow::BaseListViewBuilder< TYPE >
Public Functions
- inlineVarLengthListLikeBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&value_builder,conststd::shared_ptr<DataType>&type,int64_talignment=kDefaultBufferAlignment)#
Use this constructor to incrementally build the value array along with offsets and null bitmap.
- inlinevirtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlinevirtualvoidReset()override#
Reset the builder.
- inlineStatusAppend(boolis_valid,int64_tlist_length)#
Start a new variable-length list slot.
This function should be called before appending elements to the value builder. Elements appended to the value builder before this function is called for the first time, will not be members of any list value.
After this function is called, list_length elements SHOULD be appended to the values builder. If this contract is violated, the behavior is defined by the concrete builder implementation and SHOULD NOT be relied upon unless the caller is specifically building a [Large]List or [Large]ListView array.
For [Large]List arrays, the list slot length will be the number of elements appended to the values builder before the next call to Append* or Finish. For [Large]ListView arrays, the list slot length will be exactly list_length, but if Append* is called before at least list_length elements are appended to the values builder, the current list slot will share elements with the next list slots or an invalid [Large]ListView array will be generated because there aren’t enough elements in the values builder to fill the list slots.
If you’re building a [Large]List and don’t need to be compatible with [Large]ListView, then
BaseListBuilder::Append(boolis_valid)is a simpler API.- Parameters:
is_valid – Whether the new list slot is valid
list_length – The number of elements in the list
- Pre:
if is_valid is false, list_length MUST be 0
- inlinevirtualStatusAppendEmptyValue()final#
Append an empty list slot.
- Post:
Another call to Append* or Finish should be made before appending to the values builder to ensure list slot remains empty
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append an empty list slot.
- Post:
Another call to Append* or Finish should be made before appending to the values builder to ensure the last list slot remains empty
- virtualStatusAppendValues(constoffset_type*offsets,constoffset_type*sizes,int64_tlength,constuint8_t*valid_bytes)=0#
Vector append.
For list-array builders, the sizes are inferred from the offsets. BaseListBuilder<T> provides an implementation that doesn’t take sizes, but this virtual function allows dispatching calls to both list-array and list-view-array builders (which need the sizes)
- Parameters:
offsets – The offsets of the variable-length lists
sizes – The sizes of the variable-length lists
length – The number of offsets, sizes, and validity bits to append
valid_bytes – If passed, valid_bytes is of equal length to values, and any zero byte will be considered as a null for that slot
- inlineVarLengthListLikeBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&value_builder,conststd::shared_ptr<DataType>&type,int64_talignment=kDefaultBufferAlignment)#
- template<typenameTYPE>
classBaseListBuilder:publicarrow::VarLengthListLikeBuilder<TYPE># - #include <arrow/array/builder_nested.h>
Public Functions
- inlineStatusAppend(boolis_valid=true)#
Start a new variable-length list slot.
This function should be called before beginning to append elements to the value builder
- inlineStatusAppendValues(constoffset_type*offsets,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Vector append.
If passed, valid_bytes is of equal length to values, and any zero byte will be considered as a null for that slot
- inlinevirtualStatusAppendValues(constoffset_type*offsets,constoffset_type*sizes,int64_tlength,constuint8_t*valid_bytes)final#
Vector append.
For list-array builders, the sizes are inferred from the offsets. BaseListBuilder<T> provides an implementation that doesn’t take sizes, but this virtual function allows dispatching calls to both list-array and list-view-array builders (which need the sizes)
- Parameters:
offsets – The offsets of the variable-length lists
sizes – The sizes of the variable-length lists
length – The number of offsets, sizes, and validity bits to append
valid_bytes – If passed, valid_bytes is of equal length to values, and any zero byte will be considered as a null for that slot
- inlinevirtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- inlineStatusAppend(boolis_valid,int64_tlist_length)#
Start a new variable-length list slot.
This function should be called before appending elements to the value builder. Elements appended to the value builder before this function is called for the first time, will not be members of any list value.
After this function is called, list_length elements SHOULD be appended to the values builder. If this contract is violated, the behavior is defined by the concrete builder implementation and SHOULD NOT be relied upon unless the caller is specifically building a [Large]List or [Large]ListView array.
For [Large]List arrays, the list slot length will be the number of elements appended to the values builder before the next call to Append* or Finish. For [Large]ListView arrays, the list slot length will be exactly list_length, but if Append* is called before at least list_length elements are appended to the values builder, the current list slot will share elements with the next list slots or an invalid [Large]ListView array will be generated because there aren’t enough elements in the values builder to fill the list slots.
If you’re building a [Large]List and don’t need to be compatible with [Large]ListView, then
BaseListBuilder::Append(boolis_valid)is a simpler API.- Parameters:
is_valid – Whether the new list slot is valid
list_length – The number of elements in the list
- Pre:
if is_valid is false, list_length MUST be 0
- inlineStatusAppend(boolis_valid=true)#
- classListBuilder:publicarrow::BaseListBuilder<ListType>#
- #include <arrow/array/builder_nested.h>
Builder class for variable-length list array value types.
To use this class, you must append values to the child array builder and use the Append function to delimit each distinct list value (once the values have been appended to the child array) or use the bulk API to append a sequence of offsets and null values.
A note on types. Per arrow/type.h all types in the c++ implementation are logical so even though this class always builds list array, this can represent multiple different logical types. If no logical type is provided at construction time, the class defaults to List<T> where t is taken from the value_builder/values that the object is constructed with.
- classLargeListBuilder:publicarrow::BaseListBuilder<LargeListType>#
- #include <arrow/array/builder_nested.h>
Builder class for large variable-length list array value types.
LikeListBuilder, but to create large list arrays (with 64-bit offsets).
- template<typenameTYPE>
classBaseListViewBuilder:publicarrow::VarLengthListLikeBuilder<TYPE># - #include <arrow/array/builder_nested.h>
Public Functions
- inlinevirtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- inlinevirtualvoidReset()override#
Reset the builder.
- inlinevirtualStatusResize(int64_tcapacity)override#
- classListViewBuilder:publicarrow::BaseListViewBuilder<ListViewType>#
- #include <arrow/array/builder_nested.h>
- classLargeListViewBuilder:publicarrow::BaseListViewBuilder<LargeListViewType>#
- #include <arrow/array/builder_nested.h>
- classMapBuilder:publicarrow::ArrayBuilder#
- #include <arrow/array/builder_nested.h>
Builder class for arrays of variable-size maps.
To use this class, you must use the Append function to delimit each distinct map before appending values to the key and item array builders, or use the bulk API to append a sequence of offsets and null maps.
Key uniqueness and ordering are not validated.
Public Functions
- MapBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&key_builder,conststd::shared_ptr<ArrayBuilder>&item_builder,conststd::shared_ptr<DataType>&type)#
Use this constructor to define the built array’s type explicitly.
If key_builder or item_builder has indeterminate type, this builder will also.
- MapBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&key_builder,conststd::shared_ptr<ArrayBuilder>&item_builder,boolkeys_sorted=false)#
Use this constructor to infer the built array’s type.
If key_builder or item_builder has indeterminate type, this builder will also.
- virtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- virtualvoidReset()override#
Reset the builder.
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- StatusAppendValues(constint32_t*offsets,int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Vector append.
If passed, valid_bytes is of equal length to values, and any zero byte will be considered as a null for that slot
- StatusAppend()#
Start a new variable-length map slot.
This function should be called before beginning to append elements to the key and item builders
- virtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- virtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- inlineArrayBuilder*key_builder()const#
Get builder to append keys.
Append a key with this builder should be followed by appending an item or null value withitem_builder().
- inlineArrayBuilder*item_builder()const#
Get builder to append items.
Appending an item with this builder should have been preceded by appending a key withkey_builder().
- inlineArrayBuilder*value_builder()const#
Get builder to add Map entries as struct values.
This is used instead ofkey_builder()/item_builder() and allows the Map to be built as a list of struct values.
- MapBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&key_builder,conststd::shared_ptr<ArrayBuilder>&item_builder,conststd::shared_ptr<DataType>&type)#
- classFixedSizeListBuilder:publicarrow::ArrayBuilder#
- #include <arrow/array/builder_nested.h>
Builder class for fixed-length list array value types.
Public Functions
- FixedSizeListBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&value_builder,int32_tlist_size)#
Use this constructor to define the built array’s type explicitly.
If value_builder has indeterminate type, this builder will also.
- FixedSizeListBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&value_builder,conststd::shared_ptr<DataType>&type)#
Use this constructor to infer the built array’s type.
If value_builder has indeterminate type, this builder will also.
- virtualStatusResize(int64_tcapacity)override#
Ensure that enough memory has been allocated to fit the indicated number of total elements in the builder, including any that have already been appended.
Does not account for reallocations that may be due to variable size data, like binary values. To make space for incremental appends, use Reserve instead.
- Parameters:
capacity –[in] the minimum number of total array values to accommodate. Must be greater than the current capacity.
- Returns:
- virtualvoidReset()override#
Reset the builder.
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- StatusAppend()#
Append a valid fixed length list.
This function affects only the validity bitmap; the child values must be appended using the child array builder.
- StatusAppendValues(int64_tlength,constuint8_t*valid_bytes=NULLPTR)#
Vector append.
If passed, valid_bytes will be read and any zero byte will cause the corresponding slot to be null
This function affects only the validity bitmap; the child values must be appended using the child array builder. This includes appending nulls for null lists. XXX this restriction is confusing, should this method be omitted?
- virtualStatusAppendNull()final#
Append a null fixed length list.
The child array builder will have the appropriate number of nulls appended automatically.
- virtualStatusAppendNulls(int64_tlength)final#
Append length null fixed length lists.
The child array builder will have the appropriate number of nulls appended automatically.
- virtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- virtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- FixedSizeListBuilder(MemoryPool*pool,conststd::shared_ptr<ArrayBuilder>&value_builder,int32_tlist_size)#
- classStructBuilder:publicarrow::ArrayBuilder#
- #include <arrow/array/builder_nested.h>
Append, Resize and Reserve methods are acting onStructBuilder.
Please make sure all these methods of all child-builders’ are consistently called to maintain data-structure consistency.
Public Functions
- StructBuilder(conststd::shared_ptr<DataType>&type,MemoryPool*pool,std::vector<std::shared_ptr<ArrayBuilder>>field_builders)#
If any of field_builders has indeterminate type, this builder will also.
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- inlineStatusAppendValues(int64_tlength,constuint8_t*valid_bytes)#
Null bitmap is of equal length to every child field, and any zero byte will be considered as a null for that field, but users must using app- end methods or advance methods of the child builders’ independently to insert data.
- inlineStatusAppend(boolis_valid=true)#
Append an element to the Struct.
All child-builders’ Append method must be called independently to maintain data-structure consistency.
- inlinevirtualStatusAppendNull()final#
Append a null value.
Automatically appends an empty value to each child builder.
- inlinevirtualStatusAppendNulls(int64_tlength)final#
Append multiple null values.
Automatically appends empty values to each child builder.
- inlinevirtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlinevirtualStatusAppendArraySlice(constArraySpan&array,int64_toffset,int64_tlength)override#
Append a range of values from an array.
The given array must be the same type as the builder.
- virtualvoidReset()override#
Reset the builder.
- StructBuilder(conststd::shared_ptr<DataType>&type,MemoryPool*pool,std::vector<std::shared_ptr<ArrayBuilder>>field_builders)#
- classBasicUnionBuilder:publicarrow::ArrayBuilder#
- #include <arrow/array/builder_union.h>
Base class for union array builds.
Note that while we subclassArrayBuilder, as union types do not have a validity bitmap, the bitmap builder member ofArrayBuilder is not used.
Subclassed byarrow::DenseUnionBuilder,arrow::SparseUnionBuilder
Public Functions
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
Return result of builder as an internal genericArrayData object.
Resets builder except for dictionary builder
- int8_tAppendChild(conststd::shared_ptr<ArrayBuilder>&new_child,conststd::string&field_name="")#
Make a new child builder available to theUnionArray.
- Parameters:
new_child –[in] the child builder
field_name –[in] the name of the field in the union array type if type inference is used
- Returns:
child index, which is the “type” argument that needs to be passed to the “Append” method to add a new element to the union array.
- virtualStatusFinishInternal(std::shared_ptr<ArrayData>*out)override#
- classDenseUnionBuilder:publicarrow::BasicUnionBuilder#
- #include <arrow/array/builder_union.h>
This API is EXPERIMENTAL.
Public Functions
- inlineexplicitDenseUnionBuilder(MemoryPool*pool,int64_talignment=kDefaultBufferAlignment)#
Use this constructor to initialize the UnionBuilder with no child builders, allowing type to be inferred.
You will need to call AppendChild for each of the children builders you want to use.
- inlineDenseUnionBuilder(MemoryPool*pool,conststd::vector<std::shared_ptr<ArrayBuilder>>&children,conststd::shared_ptr<DataType>&type,int64_talignment=kDefaultBufferAlignment)#
Use this constructor to specify the type explicitly.
You can still add child builders to the union after using this constructor
- inlinevirtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlineStatusAppend(int8_tnext_type)#
Append an element to theUnionArray.
This must be followed by an append to the appropriate child builder.
The corresponding child builder must be appended to independently after this method is called.
- Parameters:
next_type –[in] type_id of the child to which the next value will be appended.
- inlineexplicitDenseUnionBuilder(MemoryPool*pool,int64_talignment=kDefaultBufferAlignment)#
- classSparseUnionBuilder:publicarrow::BasicUnionBuilder#
- #include <arrow/array/builder_union.h>
This API is EXPERIMENTAL.
Public Functions
- inlineexplicitSparseUnionBuilder(MemoryPool*pool,int64_talignment=kDefaultBufferAlignment)#
Use this constructor to initialize the UnionBuilder with no child builders, allowing type to be inferred.
You will need to call AppendChild for each of the children builders you want to use.
- inlineSparseUnionBuilder(MemoryPool*pool,conststd::vector<std::shared_ptr<ArrayBuilder>>&children,conststd::shared_ptr<DataType>&type,int64_talignment=kDefaultBufferAlignment)#
Use this constructor to specify the type explicitly.
You can still add child builders to the union after using this constructor
- inlinevirtualStatusAppendNull()final#
Append a null value.
A null is appended to the first child, empty values to the other children.
- inlinevirtualStatusAppendNulls(int64_tlength)final#
Append multiple null values.
Nulls are appended to the first child, empty values to the other children.
- inlinevirtualStatusAppendEmptyValue()final#
Append a non-null value to builder.
The appended value is an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending a null value to a parent nested type.
- inlinevirtualStatusAppendEmptyValues(int64_tlength)final#
Append a number of non-null values to builder.
The appended values are an implementation detail, but the corresponding memory slot is guaranteed to be initialized. This method is useful when appending null values to a parent nested type.
- inlineStatusAppend(int8_tnext_type)#
Append an element to theUnionArray.
This must be followed by an append to the appropriate child builder.
The corresponding child builder must be appended to independently after this method is called, and all other child builders must have null or empty value appended.
- Parameters:
next_type –[in] type_id of the child to which the next value will be appended.
- inlineexplicitSparseUnionBuilder(MemoryPool*pool,int64_talignment=kDefaultBufferAlignment)#
Dictionary-encoded#
- template<typenameT>
classDictionaryBuilder:publicarrow::internal::DictionaryBuilderBase<AdaptiveIntBuilder,T># ADictionaryArray builder that usesAdaptiveIntBuilder to return the smallest index size that can accommodate the dictionary indices.
Record Batch Builder#
- classRecordBatchBuilder#
Helper class for creating record batches iteratively given a known schema.
Public Functions
- inlineArrayBuilder*GetField(inti)#
Get base pointer to field builder.
- Parameters:
i – the field index
- Returns:
pointer toArrayBuilder
- template<typenameT>
inlineT*GetFieldAs(inti)# Return field builder casted to indicated specific builder type.
- Parameters:
i – the field index
- Returns:
pointer to template type
- Result<std::shared_ptr<RecordBatch>>Flush(boolreset_builders)#
Finish current batch and optionally reset.
- Parameters:
reset_builders –[in] the resultingRecordBatch
- Returns:
the resultingRecordBatch
- Result<std::shared_ptr<RecordBatch>>Flush()#
Finish current batch and reset.
- Returns:
the resultingRecordBatch
- voidSetInitialCapacity(int64_tcapacity)#
Set the initial capacity for new builders.
- inlineint64_tinitial_capacity()const#
The initial capacity for builders.
- inlineintnum_fields()const#
The number of fields in the schema.
Public Static Functions
- staticResult<std::unique_ptr<RecordBatchBuilder>>Make(conststd::shared_ptr<Schema>&schema,MemoryPool*pool)#
Create and initialize aRecordBatchBuilder.
- Parameters:
schema –[in] The schema for the record batch
pool –[in] AMemoryPool to use for allocations
- Returns:
the created builder instance
- staticResult<std::unique_ptr<RecordBatchBuilder>>Make(conststd::shared_ptr<Schema>&schema,MemoryPool*pool,int64_tinitial_capacity)#
Create and initialize aRecordBatchBuilder.
- Parameters:
schema –[in] The schema for the record batch
pool –[in] AMemoryPool to use for allocations
initial_capacity –[in] The initial capacity for the builders
- Returns:
the created builder instance
- inlineArrayBuilder*GetField(inti)#