Buffer structure¶

Buffer structures (or simply “buffers”) are useful as a way to expose thebinary data from another object to the Python programmer. They can also beused as a zero-copy slicing mechanism. Using their ability to reference ablock of memory, it is possible to expose any data to the Python programmerquite easily. The memory could be a large, constant array in a C extension,it could be a raw block of memory for manipulation before passing to anoperating system library, or it could be used to pass around structured datain its native, in-memory format.

Contrary to most data types exposed by the Python interpreter, buffersare notPyObject pointers but rather simple C structures. Thisallows them to be created and copied very simply. When a generic wrapperaround a buffer is needed, amemoryview objectcan be created.

For short instructions how to write an exporting object, seeBuffer Object Structures. For obtaininga buffer, seePyObject_GetBuffer().

Py_buffer¶

void *buf¶

A pointer to the start of the logical structure described by the bufferfields. This can be any location within the underlying physical memoryblock of the exporter. For example, with negativestridesthe value may point to the end of the memory block.

Forcontiguous arrays, the value points to the beginning ofthe memory block.

void *obj¶

A new reference to the exporting object. The reference is owned bythe consumer and automatically decremented and set toNULL byPyBuffer_Release(). The field is the equivalent of the returnvalue of any standard C-API function.

As a special case, fortemporary buffers that are wrapped byPyMemoryView_FromBuffer() orPyBuffer_FillInfo()this field isNULL. In general, exporting objects MUST NOTuse this scheme.

Py_ssize_tlen¶

product(shape)*itemsize. For contiguous arrays, this is the lengthof the underlying memory block. For non-contiguous arrays, it is the lengththat the logical structure would have if it were copied to a contiguousrepresentation.

Accessing((char*)buf)[0]upto((char*)buf)[len-1] is only validif the buffer has been obtained by a request that guarantees contiguity. Inmost cases such a request will bePyBUF_SIMPLE orPyBUF_WRITABLE.

intreadonly¶

An indicator of whether the buffer is read-only. This field is controlledby thePyBUF_WRITABLE flag.

Py_ssize_titemsize¶

Item size in bytes of a single element. Same as the value ofstruct.calcsize()called on non-NULLformat values.

Important exception: If a consumer requests a buffer without thePyBUF_FORMAT flag,format willbe set toNULL, butitemsize still hasthe value for the original format.

Ifshape is present, the equalityproduct(shape)*itemsize==len still holds and the consumercan useitemsize to navigate the buffer.

Ifshape isNULL as a result of aPyBUF_SIMPLEor aPyBUF_WRITABLE request, the consumer must disregarditemsize and assumeitemsize==1.

const char *format¶

ANUL terminated string instruct module style syntax describingthe contents of a single item. If this isNULL,"B" (unsigned bytes)is assumed.

This field is controlled by thePyBUF_FORMAT flag.

intndim¶

The number of dimensions the memory represents as an n-dimensional array.If it is0,buf points to a single item representinga scalar. In this case,shape,stridesandsuboffsets MUST beNULL.

The macroPyBUF_MAX_NDIM limits the maximum number of dimensionsto 64. Exporters MUST respect this limit, consumers of multi-dimensionalbuffers SHOULD be able to handle up toPyBUF_MAX_NDIM dimensions.

Py_ssize_t *shape¶

An array ofPy_ssize_t of lengthndimindicating the shape of the memory as an n-dimensional array. Note thatshape[0]*...*shape[ndim-1]*itemsize MUST be equal tolen.

Shape values are restricted toshape[n]>=0. The caseshape[n]==0 requires special attention. Seecomplex arraysfor further information.

The shape array is read-only for the consumer.

Py_ssize_t *strides¶

An array ofPy_ssize_t of lengthndimgiving the number of bytes to skip to get to a new element in eachdimension.

Stride values can be any integer. For regular arrays, strides areusually positive, but a consumer MUST be able to handle the casestrides[n]<=0. Seecomplex arrays for further information.

The strides array is read-only for the consumer.

Py_ssize_t *suboffsets¶

An array ofPy_ssize_t of lengthndim.Ifsuboffsets[n]>=0, the values stored along the nth dimension arepointers and the suboffset value dictates how many bytes to add to eachpointer after de-referencing. A suboffset value that is negativeindicates that no de-referencing should occur (striding in a contiguousmemory block).

If all suboffsets are negative (i.e. no de-referencing is needed), thenthis field must beNULL (the default value).

This type of array representation is used by the Python Imaging Library(PIL). Seecomplex arrays for further information how to access elementsof such an array.

The suboffsets array is read-only for the consumer.

void *internal¶

This is for use internally by the exporting object. For example, thismight be re-cast as an integer by the exporter and used to store flagsabout whether or not the shape, strides, and suboffsets arrays must befreed when the buffer is released. The consumer MUST NOT alter thisvalue.

Buffer request types¶

Buffers are usually obtained by sending a buffer request to an exportingobject viaPyObject_GetBuffer(). Since the complexity of the logicalstructure of the memory can vary drastically, the consumer uses theflagsargument to specify the exact buffer type it can handle.

AllPy_buffer fields are unambiguously defined by the requesttype.

Complex arrays¶

ptr=(char*)buf+indices[0]*strides[0]+...+indices[n-1]*strides[n-1];item=*((typeof(item)*)ptr);

defverify_structure(memlen,itemsize,ndim,shape,strides,offset):"""Verify that the parameters represent a valid array within       the bounds of the allocated memory:           char *mem: start of the physical memory block           memlen: length of the physical memory block           offset: (char *)buf - mem    """ifoffset%itemsize:returnFalseifoffset<0oroffset+itemsize>memlen:returnFalseifany(v%itemsizeforvinstrides):returnFalseifndim<=0:returnndim==0andnotshapeandnotstridesif0inshape:returnTrueimin=sum(strides[j]*(shape[j]-1)forjinrange(ndim)ifstrides[j]<=0)imax=sum(strides[j]*(shape[j]-1)forjinrange(ndim)ifstrides[j]>0)return0<=offset+iminandoffset+imax+itemsize<=memlen

void*get_item_pointer(intndim,void*buf,Py_ssize_t*strides,Py_ssize_t*suboffsets,Py_ssize_t*indices){char*pointer=(char*)buf;inti;for(i=0;i<ndim;i++){pointer+=strides[i]*indices[i];if(suboffsets[i]>=0){pointer=*((char**)pointer)+suboffsets[i];}}return(void*)pointer;}

Buffer-related functions¶

intPyObject_CheckBuffer(PyObject *obj)¶

Return1 ifobj supports the buffer interface otherwise0. When1 isreturned, it doesn’t guarantee thatPyObject_GetBuffer() willsucceed. This function always succeeds.

intPyObject_GetBuffer(PyObject *exporter,Py_buffer *view, int flags)¶

Send a request toexporter to fill inview as specified byflags.If the exporter cannot provide a buffer of the exact type, it MUST raisePyExc_BufferError, setview->obj toNULL andreturn-1.

On success, fill inview, setview->obj to a new referencetoexporter and return 0. In the case of chained buffer providersthat redirect requests to a single object,view->obj MAYrefer to this object instead ofexporter (SeeBuffer Object Structures).

Successful calls toPyObject_GetBuffer() must be paired with callstoPyBuffer_Release(), similar tomalloc() andfree().Thus, after the consumer is done with the buffer,PyBuffer_Release()must be called exactly once.

voidPyBuffer_Release(Py_buffer *view)¶

Release the bufferview and decrement the reference count forview->obj. This function MUST be called when the bufferis no longer being used, otherwise reference leaks may occur.

It is an error to call this function on a buffer that was not obtained viaPyObject_GetBuffer().

Py_ssize_tPyBuffer_SizeFromFormat(const char *format)¶

Return the implieditemsize fromformat.On error, raise an exception and return -1.

New in version 3.9.

intPyBuffer_IsContiguous(Py_buffer *view, char order)¶

Return1 if the memory defined by theview is C-style (order is'C') or Fortran-style (order is'F')contiguous or either one(order is'A'). Return0 otherwise. This function always succeeds.

void*PyBuffer_GetPointer(Py_buffer *view,Py_ssize_t *indices)¶

Get the memory area pointed to by theindices inside the givenview.indices must point to an array ofview->ndim indices.

intPyBuffer_FromContiguous(Py_buffer *view, void *buf,Py_ssize_t len, char fort)¶

Copy contiguouslen bytes frombuf toview.fort can be'C' or'F' (for C-style or Fortran-style ordering).0 is returned on success,-1 on error.

intPyBuffer_ToContiguous(void *buf,Py_buffer *src,Py_ssize_t len, char order)¶

Copylen bytes fromsrc to its contiguous representation inbuf.order can be'C' or'F' or'A' (for C-style or Fortran-styleordering or either one).0 is returned on success,-1 on error.

This function fails iflen !=src->len.

voidPyBuffer_FillContiguousStrides(int ndims,Py_ssize_t *shape,Py_ssize_t *strides, int itemsize, char order)¶

Fill thestrides array with byte-strides of acontiguous (C-style iforder is'C' or Fortran-style iforder is'F') array of thegiven shape with the given number of bytes per element.

intPyBuffer_FillInfo(Py_buffer *view,PyObject *exporter, void *buf,Py_ssize_t len, int readonly, int flags)¶

Handle buffer requests for an exporter that wants to exposebuf of sizelenwith writability set according toreadonly.buf is interpreted as a sequenceof unsigned bytes.

Theflags argument indicates the request type. This function always fills inview as specified by flags, unlessbuf has been designated as read-onlyandPyBUF_WRITABLE is set inflags.

On success, setview->obj to a new reference toexporter andreturn 0. Otherwise, raisePyExc_BufferError, setview->obj toNULL and return-1;

If this function is used as part of agetbufferproc,exporter MUST be set to the exporting object andflags must be passedunmodified. Otherwise,exporter MUST beNULL.