Data formats¶

Pillow currently supports exporting Arrow images in all modes.

For single-band images, the exported array is width*height elements,with each pixel corresponding to the appropriate Arrow type.

For multiband images, the exported array is width*height fixed-lengthfour-element arrays of uint8. This is memory compatible with the rawimage storage of four bytes per pixel.

Mode1 images are exported as one uint8 byte/pixel, as this isconsistent with the internal storage.

Pillow will accept, but not produce, one other format. For anymultichannel image with 32-bit storage per pixel, Pillow will acceptan array of width*height int32 elements, which will then beinterpreted using the mode-specific interpretation of the bytes.

The image mode must match the Arrow band format when reading singlechannel images.

Memory allocator¶

Pillow’s default memory allocator, theBlock allocator,allocates up to a 16 MB block for images by default. Larger imagesoverflow into additional blocks. Arrow requires a single continuousmemory allocation, so images allocated in multiple blocks cannot beexported in the Arrow format.

To enable the single block allocator:

fromPILimportImageImage.core.set_use_block_allocator(1)

Note that this is a global setting, not a per-image setting.

Unsupported features¶

• Table/dataframe protocol. We support a single array.

• Null markers, producing or consuming. Null values are inferred fromthe mode, e.g. RGB images are stored in the first three bytes ofeach 32-bit pixel, and the last byte is an implied null.

• Schema negotiation. There is an optional schema for the requesteddatatype in the Arrow source interface. We ignore thatparameter.

• Array metadata.

Internal details¶

Python Arrow C interface:https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html

The memory that is exported from the Arrow interface is shared – notcopied, so the lifetime of the memory allocation is no longer strictlytied to the life of the Python object.

The core imaging struct now has a refcount associated with it, and thelifetime of the core image struct is now divorced from the Pythonimage object. Creating an arrow reference to the image increments therefcount, and the imaging struct is only released when the refcountreaches zero.