Structured datatype creation#

Structured datatypes may be created using the functionnumpy.dtype.There are 4 alternative forms of specification which vary in flexibility andconciseness. These are further documented in theData Type Objects reference page, and insummary they are:

1. A list of tuples, one tuple per field

   Each tuple has the form(fieldname,datatype,shape) where shape isoptional.fieldname is a string (or tuple if titles are used, seeField Titles below),datatype may be any objectconvertible to a datatype, andshape is a tuple of integers specifyingsubarray shape.

   >>>np.dtype([('x','f4'),('y',np.float32),('z','f4',(2,2))])dtype([('x', '<f4'), ('y', '<f4'), ('z', '<f4', (2, 2))])

   Iffieldname is the empty string'', the field will be given adefault name of the formf#, where# is the integer index of thefield, counting from 0 from the left:

   >>>np.dtype([('x','f4'),('','i4'),('z','i8')])dtype([('x', '<f4'), ('f1', '<i4'), ('z', '<i8')])

   The byte offsets of the fields within the structure and the totalstructure itemsize are determined automatically.

2. A string of comma-separated dtype specifications

   In this shorthand notation any of thestring dtype specifications may be used in a string and separated bycommas. The itemsize and byte offsets of the fields are determinedautomatically, and the field names are given the default namesf0,f1, etc.

   >>>np.dtype('i8, f4, S3')dtype([('f0', '<i8'), ('f1', '<f4'), ('f2', 'S3')])>>>np.dtype('3int8, float32, (2, 3)float64')dtype([('f0', 'i1', (3,)), ('f1', '<f4'), ('f2', '<f8', (2, 3))])

3. A dictionary of field parameter arrays

   This is the most flexible form of specification since it allows controlover the byte-offsets of the fields and the itemsize of the structure.

   The dictionary has two required keys, ‘names’ and ‘formats’, and fouroptional keys, ‘offsets’, ‘itemsize’, ‘aligned’ and ‘titles’. The valuesfor ‘names’ and ‘formats’ should respectively be a list of field names anda list of dtype specifications, of the same length. The optional ‘offsets’value should be a list of integer byte-offsets, one for each field withinthe structure. If ‘offsets’ is not given the offsets are determinedautomatically. The optional ‘itemsize’ value should be an integerdescribing the total size in bytes of the dtype, which must be largeenough to contain all the fields.

   >>>np.dtype({'names':['col1','col2'],'formats':['i4','f4']})dtype([('col1', '<i4'), ('col2', '<f4')])>>>np.dtype({'names':['col1','col2'],...'formats':['i4','f4'],...'offsets':[0,4],...'itemsize':12})dtype({'names': ['col1', 'col2'], 'formats': ['<i4', '<f4'], 'offsets': [0, 4], 'itemsize': 12})

   Offsets may be chosen such that the fields overlap, though this will meanthat assigning to one field may clobber any overlapping field’s data. Asan exception, fields ofnumpy.object_ type cannot overlap withother fields, because of the risk of clobbering the internal objectpointer and then dereferencing it.

   The optional ‘aligned’ value can be set toTrue to make the automaticoffset computation use aligned offsets (seeAutomatic byte offsets and alignment),as if the ‘align’ keyword argument ofnumpy.dtype had been set toTrue.

   The optional ‘titles’ value should be a list of titles of the same lengthas ‘names’, seeField Titles below.

4. A dictionary of field names

   The keys of the dictionary are the field names and the values are tuplesspecifying type and offset:

   >>>np.dtype({'col1':('i1',0),'col2':('f4',1)})dtype([('col1', 'i1'), ('col2', '<f4')])

   This form was discouraged because Python dictionaries did not preserve orderin Python versions before Python 3.6.Field Titles may bespecified by using a 3-tuple, see below.

Manipulating and displaying structured datatypes#

The list of field names of a structured datatype can be found in thenamesattribute of the dtype object:

>>>d=np.dtype([('x','i8'),('y','f4')])>>>d.names('x', 'y')

The dtype of each individual field can be looked up by name:

>>>d['x']dtype('int64')

The field names may be modified by assigning to thenames attribute using asequence of strings of the same length.

The dtype object also has a dictionary-like attribute,fields, whose keysare the field names (andField Titles, see below) and whosevalues are tuples containing the dtype and byte offset of each field.

>>>d.fieldsmappingproxy({'x': (dtype('int64'), 0), 'y': (dtype('float32'), 8)})

Both thenames andfields attributes will equalNone forunstructured arrays. The recommended way to test if a dtype is structured iswithif dt.names is not None rather thanif dt.names, to account for dtypeswith 0 fields.

The string representation of a structured datatype is shown in the “list oftuples” form if possible, otherwise numpy falls back to using the more generaldictionary form.

Automatic byte offsets and alignment#

Numpy uses one of two methods to automatically determine the field byte offsetsand the overall itemsize of a structured datatype, depending on whetheralign=True was specified as a keyword argument tonumpy.dtype.

By default (align=False), numpy will pack the fields together such thateach field starts at the byte offset the previous field ended, and the fieldsare contiguous in memory.

>>>defprint_offsets(d):...print("offsets:",[d.fields[name][1]fornameind.names])...print("itemsize:",d.itemsize)>>>print_offsets(np.dtype('u1, u1, i4, u1, i8, u2'))offsets: [0, 1, 2, 6, 7, 15]itemsize: 17

Ifalign=True is set, numpy will pad the structure in the same way many Ccompilers would pad a C-struct. Aligned structures can give a performanceimprovement in some cases, at the cost of increased datatype size. Paddingbytes are inserted between fields such that each field’s byte offset will be amultiple of that field’s alignment, which is usually equal to the field’s sizein bytes for simple datatypes, seePyArray_Descr.alignment. Thestructure will also have trailing padding added so that its itemsize is amultiple of the largest field’s alignment.

>>>print_offsets(np.dtype('u1, u1, i4, u1, i8, u2',align=True))offsets: [0, 1, 4, 8, 16, 24]itemsize: 32

Note that although almost all modern C compilers pad in this way by default,padding in C structs is C-implementation-dependent so this memory layout is notguaranteed to exactly match that of a corresponding struct in a C program. Somework may be needed, either on the numpy side or the C side, to obtain exactcorrespondence.

If offsets were specified using the optionaloffsets key in thedictionary-based dtype specification, settingalign=True will check thateach field’s offset is a multiple of its size and that the itemsize is amultiple of the largest field size, and raise an exception if not.

If the offsets of the fields and itemsize of a structured array satisfy thealignment conditions, the array will have theALIGNEDflag set.

A convenience functionnumpy.lib.recfunctions.repack_fields converts analigned dtype or array to a packed one and vice versa. It takes either a dtypeor structured ndarray as an argument, and returns a copy with fields re-packed,with or without padding bytes.

Field titles#

In addition to field names, fields may also have an associatedtitle,an alternate name, which is sometimes used as an additional description oralias for the field. The title may be used to index an array, just like afield name.

To add titles when using the list-of-tuples form of dtype specification, thefield name may be specified as a tuple of two strings instead of a singlestring, which will be the field’s title and field name respectively. Forexample:

>>>np.dtype([(('my title','name'),'f4')])dtype([(('my title', 'name'), '<f4')])

When using the first form of dictionary-based specification, the titles may besupplied as an extra'titles' key as described above. When using the second(discouraged) dictionary-based specification, the title can be supplied byproviding a 3-element tuple(datatype,offset,title) instead of the usual2-element tuple:

>>>np.dtype({'name':('i4',0,'my title')})dtype([(('my title', 'name'), '<i4')])

Thedtype.fields dictionary will contain titles as keys, if anytitles are used. This means effectively that a field with a title will berepresented twice in the fields dictionary. The tuple values for these fieldswill also have a third element, the field title. Because of this, and becausethenames attribute preserves the field order while thefieldsattribute may not, it is recommended to iterate through the fields of a dtypeusing thenames attribute of the dtype, which will not list titles, asin:

>>>fornameind.names:...print(d.fields[name][:2])(dtype('int64'), 0)(dtype('float32'), 8)

Union types#

Structured datatypes are implemented in numpy to have base typenumpy.void by default, but it is possible to interpret other numpytypes as structured types using the(base_dtype,dtype) form of dtypespecification described inData Type Objects. Here,base_dtype isthe desired underlying dtype, and fields and flags will be copied fromdtype. This dtype is similar to a ‘union’ in C.

Assigning data to a structured array#

There are a number of ways to assign values to a structured array: Using pythontuples, using scalar values, or using other structured arrays.

>>>x=np.array([(1,2,3),(4,5,6)],dtype='i8, f4, f8')>>>x[1]=(7,8,9)>>>xarray([(1, 2., 3.), (7, 8., 9.)],     dtype=[('f0', '<i8'), ('f1', '<f4'), ('f2', '<f8')])

>>>x=np.zeros(2,dtype='i8, f4, ?, S1')>>>x[:]=3>>>xarray([(3, 3., True, b'3'), (3, 3., True, b'3')],      dtype=[('f0', '<i8'), ('f1', '<f4'), ('f2', '?'), ('f3', 'S1')])>>>x[:]=np.arange(2)>>>xarray([(0, 0., False, b'0'), (1, 1., True, b'1')],      dtype=[('f0', '<i8'), ('f1', '<f4'), ('f2', '?'), ('f3', 'S1')])

>>>twofield=np.zeros(2,dtype=[('A','i4'),('B','i4')])>>>onefield=np.zeros(2,dtype=[('A','i4')])>>>nostruct=np.zeros(2,dtype='i4')>>>nostruct[:]=twofieldTraceback (most recent call last):...TypeError:Cannot cast array data from dtype([('A', '<i4'), ('B', '<i4')]) to dtype('int32') according to the rule 'unsafe'

>>>a=np.zeros(3,dtype=[('a','i8'),('b','f4'),('c','S3')])>>>b=np.ones(3,dtype=[('x','f4'),('y','S3'),('z','O')])>>>b[:]=a>>>barray([(0., b'0.0', b''), (0., b'0.0', b''), (0., b'0.0', b'')],      dtype=[('x', '<f4'), ('y', 'S3'), ('z', 'O')])

Indexing structured arrays#

>>>x=np.array([(1,2),(3,4)],dtype=[('foo','i8'),('bar','f4')])>>>x['foo']array([1, 3])>>>x['foo']=10>>>xarray([(10, 2.), (10, 4.)],      dtype=[('foo', '<i8'), ('bar', '<f4')])

>>>y=x['bar']>>>y[:]=11>>>xarray([(10, 11.), (10, 11.)],      dtype=[('foo', '<i8'), ('bar', '<f4')])

>>>y.dtype,y.shape,y.strides(dtype('float32'), (2,), (12,))

>>>x=np.zeros((2,2),dtype=[('a',np.int32),('b',np.float64,(3,3))])>>>x['a'].shape(2, 2)>>>x['b'].shape(2, 2, 3, 3)

>>>a=np.zeros(3,dtype=[('a','i4'),('b','i4'),('c','f4')])>>>a[['a','c']]array([(0, 0.), (0, 0.), (0, 0.)],     dtype={'names': ['a', 'c'], 'formats': ['<i4', '<f4'], 'offsets': [0, 8], 'itemsize': 12})

>>>a[['a','c']].view('i8')# Fails in Numpy 1.16Traceback (most recent call last):   File "<stdin>", line 1, in <module>ValueError:When changing to a smaller dtype, its size must be a divisor of the size of original dtype

>>>fromnumpy.lib.recfunctionsimportrepack_fields>>>repack_fields(a[['a','c']]).view('i8')# supported in 1.16array([0, 0, 0])

>>>b=np.zeros(3,dtype=[('x','f4'),('y','f4'),('z','f4')])>>>b[['x','z']].view('f4')array([0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)

>>>fromnumpy.lib.recfunctionsimportstructured_to_unstructured>>>structured_to_unstructured(b[['x','z']])array([[0., 0.],       [0., 0.],       [0., 0.]], dtype=float32)

>>>a[['a','c']]=(2,3)>>>aarray([(2, 0, 3.), (2, 0, 3.), (2, 0, 3.)],      dtype=[('a', '<i4'), ('b', '<i4'), ('c', '<f4')])

>>>a[['a','c']]=a[['c','a']]

>>>x=np.array([(1,2.,3.)],dtype='i, f, f')>>>scalar=x[0]>>>scalarnp.void((1, 2.0, 3.0), dtype=[('f0', '<i4'), ('f1', '<f4'), ('f2', '<f4')])>>>type(scalar)<class 'numpy.void'>

>>>x=np.array([(1,2),(3,4)],dtype=[('foo','i8'),('bar','f4')])>>>s=x[0]>>>s['bar']=100>>>xarray([(1, 100.), (3, 4.)],      dtype=[('foo', '<i8'), ('bar', '<f4')])

>>>scalar=np.array([(1,2.,3.)],dtype='i, f, f')[0]>>>scalar[0]np.int32(1)>>>scalar[1]=4

>>>scalar.item(),type(scalar.item())((1, 4.0, 3.0), <class 'tuple'>)

Viewing structured arrays containing objects#

In order to prevent clobbering object pointers in fields ofobject type, numpy currently does not allow views of structuredarrays containing objects.

Structure comparison and promotion#

If the dtypes of two void structured arrays are equal, testing the equality ofthe arrays will result in a boolean array with the dimensions of the originalarrays, with elements set toTrue where all fields of the correspondingstructures are equal:

>>>a=np.array([(1,1),(2,2)],dtype=[('a','i4'),('b','i4')])>>>b=np.array([(1,1),(2,3)],dtype=[('a','i4'),('b','i4')])>>>a==barray([True, False])

NumPy will promote individual field datatypes to perform the comparison.So the following is also valid (note the'f4' dtype for the'a' field):

>>>b=np.array([(1.0,1),(2.5,2)],dtype=[("a","f4"),("b","i4")])>>>a==barray([True, False])

To compare two structured arrays, it must be possible to promote them to acommon dtype as returned bynumpy.result_type andnumpy.promote_types.This enforces that the number of fields, the field names, and the field titlesmust match precisely.When promotion is not possible, for example due to mismatching field names,NumPy will raise an error.Promotion between two structured dtypes results in a canonical dtype thatensures native byte-order for all fields:

>>>np.result_type(np.dtype("i,>i"))dtype([('f0', '<i4'), ('f1', '<i4')])>>>np.result_type(np.dtype("i,>i"),np.dtype("i,i"))dtype([('f0', '<i4'), ('f1', '<i4')])

The resulting dtype from promotion is also guaranteed to be packed, meaningthat all fields are ordered contiguously and any unnecessary padding isremoved:

>>>dt=np.dtype("i1,V3,i4,V1")[["f0","f2"]]>>>dtdtype({'names': ['f0', 'f2'], 'formats': ['i1', '<i4'], 'offsets': [0, 4], 'itemsize': 9})>>>np.result_type(dt)dtype([('f0', 'i1'), ('f2', '<i4')])

Note that the result prints withoutoffsets oritemsize indicating noadditional padding.If a structured dtype is created withalign=True ensuring thatdtype.isalignedstruct is true, this property is preserved:

>>>dt=np.dtype("i1,V3,i4,V1",align=True)[["f0","f2"]]>>>dtdtype({'names': ['f0', 'f2'], 'formats': ['i1', '<i4'], 'offsets': [0, 4], 'itemsize': 12}, align=True)>>>np.result_type(dt)dtype([('f0', 'i1'), ('f2', '<i4')], align=True)>>>np.result_type(dt).isalignedstructTrue

When promoting multiple dtypes, the result is aligned if any of the inputs is:

>>>np.result_type(np.dtype("i,i"),np.dtype("i,i",align=True))dtype([('f0', '<i4'), ('f1', '<i4')], align=True)

The< and> operators always returnFalse when comparing voidstructured arrays, and arithmetic and bitwise operations are not supported.

Recarray helper functions#

Collection of utilities to manipulate structured arrays.

Most of these functions were initially implemented by John Hunter formatplotlib. They have been rewritten and extended for convenience.

numpy.lib.recfunctions.append_fields(base,names,data,dtypes=None,fill_value=-1,usemask=True,asrecarray=False)[source]#

Add new fields to an existing array.

The names of the fields are given with thenames arguments,the corresponding values with thedata arguments.If a single field is appended,names,data anddtypes do not haveto be lists but just values.

Parameters:

basearray

Input array to extend.

namesstring, sequence

String or sequence of strings corresponding to the namesof the new fields.

dataarray or sequence of arrays

Array or sequence of arrays storing the fields to add to the base.

dtypessequence of datatypes, optional

Datatype or sequence of datatypes.If None, the datatypes are estimated from thedata.

fill_value{float}, optional

Filling value used to pad missing data on the shorter arrays.

usemask{False, True}, optional

Whether to return a masked array or not.

asrecarray{False, True}, optional

Whether to return a recarray (MaskedRecords) or not.

numpy.lib.recfunctions.apply_along_fields(func,arr)[source]#

Apply function ‘func’ as a reduction across fields of a structured array.

This is similar tonumpy.apply_along_axis, but treats the fields of astructured array as an extra axis. The fields are all first cast to acommon type following the type-promotion rules fromnumpy.result_typeapplied to the field’s dtypes.

Parameters:

funcfunction

Function to apply on the “field” dimension. This function mustsupport anaxis argument, likenumpy.mean,numpy.sum, etc.

arrndarray

Structured array for which to apply func.

Returns:

outndarray

Result of the reduction operation

Examples

Try it in your browser!

>>>importnumpyasnp

>>>fromnumpy.libimportrecfunctionsasrfn>>>b=np.array([(1,2,5),(4,5,7),(7,8,11),(10,11,12)],...dtype=[('x','i4'),('y','f4'),('z','f8')])>>>rfn.apply_along_fields(np.mean,b)array([ 2.66666667,  5.33333333,  8.66666667, 11.        ])>>>rfn.apply_along_fields(np.mean,b[['x','z']])array([ 3. ,  5.5,  9. , 11. ])

Go BackOpen In Tab

numpy.lib.recfunctions.assign_fields_by_name(dst,src,zero_unassigned=True)[source]#

Assigns values from one structured array to another by field name.

Normally in numpy >= 1.14, assignment of one structured array to anothercopies fields “by position”, meaning that the first field from the src iscopied to the first field of the dst, and so on, regardless of field name.

This function instead copies “by field name”, such that fields in the dstare assigned from the identically named field in the src. This appliesrecursively for nested structures. This is how structure assignment workedin numpy >= 1.6 to <= 1.13.

Parameters:

dstndarray

srcndarray

The source and destination arrays during assignment.

zero_unassignedbool, optional

If True, fields in the dst for which there was no matchingfield in the src are filled with the value 0 (zero). Thiswas the behavior of numpy <= 1.13. If False, those fieldsare not modified.

numpy.lib.recfunctions.drop_fields(base,drop_names,usemask=True,asrecarray=False)[source]#

Return a new array with fields indrop_names dropped.

Nested fields are supported.

Parameters:

basearray

Input array

drop_namesstring or sequence

String or sequence of strings corresponding to the names of thefields to drop.

usemask{False, True}, optional

Whether to return a masked array or not.

asrecarraystring or sequence, optional

Whether to return a recarray or a mrecarray (asrecarray=True) ora plain ndarray or masked array with flexible dtype. The defaultis False.

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>a=np.array([(1,(2,3.0)),(4,(5,6.0))],...dtype=[('a',np.int64),('b',[('ba',np.double),('bb',np.int64)])])>>>rfn.drop_fields(a,'a')array([((2., 3),), ((5., 6),)],      dtype=[('b', [('ba', '<f8'), ('bb', '<i8')])])>>>rfn.drop_fields(a,'ba')array([(1, (3,)), (4, (6,))], dtype=[('a', '<i8'), ('b', [('bb', '<i8')])])>>>rfn.drop_fields(a,['ba','bb'])array([(1,), (4,)], dtype=[('a', '<i8')])

Go BackOpen In Tab

numpy.lib.recfunctions.find_duplicates(a,key=None,ignoremask=True,return_index=False)[source]#

Find the duplicates in a structured array along a given key

Parameters:

aarray-like

Input array

key{string, None}, optional

Name of the fields along which to check the duplicates.If None, the search is performed by records

ignoremask{True, False}, optional

Whether masked data should be discarded or considered as duplicates.

return_index{False, True}, optional

Whether to return the indices of the duplicated values.

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>ndtype=[('a',int)]>>>a=np.ma.array([1,1,1,2,2,3,3],...mask=[0,0,1,0,0,0,1]).view(ndtype)>>>rfn.find_duplicates(a,ignoremask=True,return_index=True)(masked_array(data=[(1,), (1,), (2,), (2,)],             mask=[(False,), (False,), (False,), (False,)],       fill_value=(999999,),            dtype=[('a', '<i8')]), array([0, 1, 3, 4]))

Go BackOpen In Tab

numpy.lib.recfunctions.flatten_descr(ndtype)[source]#

Flatten a structured data-type description.

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>ndtype=np.dtype([('a','<i4'),('b',[('ba','<f8'),('bb','<i4')])])>>>rfn.flatten_descr(ndtype)(('a', dtype('int32')), ('ba', dtype('float64')), ('bb', dtype('int32')))

Go BackOpen In Tab

numpy.lib.recfunctions.get_fieldstructure(adtype,lastname=None,parents=None)[source]#

Returns a dictionary with fields indexing lists of their parent fields.

This function is used to simplify access to fields nested in other fields.

Parameters:

adtypenp.dtype

Input datatype

lastnameoptional

Last processed field name (used internally during recursion).

parentsdictionary

Dictionary of parent fields (used internally during recursion).

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>ndtype=np.dtype([('A',int),...('B',[('BA',int),...('BB',[('BBA',int),('BBB',int)])])])>>>rfn.get_fieldstructure(ndtype)...# XXX: possible regression, order of BBA and BBB is swapped{'A': [], 'B': [], 'BA': ['B'], 'BB': ['B'], 'BBA': ['B', 'BB'], 'BBB': ['B', 'BB']}

Go BackOpen In Tab

numpy.lib.recfunctions.get_names(adtype)[source]#

Returns the field names of the input datatype as a tuple. Input datatypemust have fields otherwise error is raised.

Parameters:

adtypedtype

Input datatype

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>rfn.get_names(np.empty((1,),dtype=[('A',int)]).dtype)('A',)>>>rfn.get_names(np.empty((1,),dtype=[('A',int),('B',float)]).dtype)('A', 'B')>>>adtype=np.dtype([('a',int),('b',[('ba',int),('bb',int)])])>>>rfn.get_names(adtype)('a', ('b', ('ba', 'bb')))

Go BackOpen In Tab

numpy.lib.recfunctions.get_names_flat(adtype)[source]#

Returns the field names of the input datatype as a tuple. Input datatypemust have fields otherwise error is raised.Nested structure are flattened beforehand.

Parameters:

adtypedtype

Input datatype

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>rfn.get_names_flat(np.empty((1,),dtype=[('A',int)]).dtype)isNoneFalse>>>rfn.get_names_flat(np.empty((1,),dtype=[('A',int),('B',str)]).dtype)('A', 'B')>>>adtype=np.dtype([('a',int),('b',[('ba',int),('bb',int)])])>>>rfn.get_names_flat(adtype)('a', 'b', 'ba', 'bb')

Go BackOpen In Tab

numpy.lib.recfunctions.join_by(key,r1,r2,jointype='inner',r1postfix='1',r2postfix='2',defaults=None,usemask=True,asrecarray=False)[source]#

Join arraysr1 andr2 on keykey.

The key should be either a string or a sequence of string correspondingto the fields used to join the array. An exception is raised if thekey field cannot be found in the two input arrays. Neitherr1 norr2 should have any duplicates alongkey: the presence of duplicateswill make the output quite unreliable. Note that duplicates are notlooked for by the algorithm.

Parameters:

key{string, sequence}

A string or a sequence of strings corresponding to the fields usedfor comparison.

r1, r2arrays

Structured arrays.

jointype{‘inner’, ‘outer’, ‘leftouter’}, optional

If ‘inner’, returns the elements common to both r1 and r2.If ‘outer’, returns the common elements as well as the elements ofr1 not in r2 and the elements of not in r2.If ‘leftouter’, returns the common elements and the elements of r1not in r2.

r1postfixstring, optional

String appended to the names of the fields of r1 that are presentin r2 but absent of the key.

r2postfixstring, optional

String appended to the names of the fields of r2 that are presentin r1 but absent of the key.

defaults{dictionary}, optional

Dictionary mapping field names to the corresponding default values.

usemask{True, False}, optional

Whether to return a MaskedArray (or MaskedRecords isasrecarray==True) or a ndarray.

asrecarray{False, True}, optional

Whether to return a recarray (or MaskedRecords ifusemask==True)or just a flexible-type ndarray.

Notes

• The output is sorted along the key.

• A temporary array is formed by dropping the fields not in the key forthe two arrays and concatenating the result. This array is thensorted, and the common entries selected. The output is constructed byfilling the fields with the selected entries. Matching is notpreserved if there are some duplicates…

numpy.lib.recfunctions.merge_arrays(seqarrays,fill_value=-1,flatten=False,usemask=False,asrecarray=False)[source]#

Merge arrays field by field.

Parameters:

seqarrayssequence of ndarrays

Sequence of arrays

fill_value{float}, optional

Filling value used to pad missing data on the shorter arrays.

flatten{False, True}, optional

Whether to collapse nested fields.

usemask{False, True}, optional

Whether to return a masked array or not.

asrecarray{False, True}, optional

Whether to return a recarray (MaskedRecords) or not.

Notes

• Without a mask, the missing value will be filled with something,depending on what its corresponding type:

  • -1 for integers

  • -1.0 for floating point numbers

  • '-' for characters

  • '-1' for strings

  • True for boolean values

• XXX: I just obtained these values empirically

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>rfn.merge_arrays((np.array([1,2]),np.array([10.,20.,30.])))array([( 1, 10.), ( 2, 20.), (-1, 30.)],      dtype=[('f0', '<i8'), ('f1', '<f8')])

>>>rfn.merge_arrays((np.array([1,2],dtype=np.int64),...np.array([10.,20.,30.])),usemask=False) array([(1, 10.0), (2, 20.0), (-1, 30.0)],         dtype=[('f0', '<i8'), ('f1', '<f8')])>>>rfn.merge_arrays((np.array([1,2]).view([('a',np.int64)]),...np.array([10.,20.,30.])),...usemask=False,asrecarray=True)rec.array([( 1, 10.), ( 2, 20.), (-1, 30.)],          dtype=[('a', '<i8'), ('f1', '<f8')])

Go BackOpen In Tab

numpy.lib.recfunctions.rec_append_fields(base,names,data,dtypes=None)[source]#

Add new fields to an existing array.

The names of the fields are given with thenames arguments,the corresponding values with thedata arguments.If a single field is appended,names,data anddtypes do not haveto be lists but just values.

Parameters:

basearray

Input array to extend.

namesstring, sequence

String or sequence of strings corresponding to the namesof the new fields.

dataarray or sequence of arrays

Array or sequence of arrays storing the fields to add to the base.

dtypessequence of datatypes, optional

Datatype or sequence of datatypes.If None, the datatypes are estimated from thedata.

Returns:

appended_arraynp.recarray

See also

append_fields

numpy.lib.recfunctions.rec_drop_fields(base,drop_names)[source]#

Returns a new numpy.recarray with fields indrop_names dropped.

numpy.lib.recfunctions.rec_join(key,r1,r2,jointype='inner',r1postfix='1',r2postfix='2',defaults=None)[source]#

Join arraysr1 andr2 on keys.Alternative to join_by, that always returns a np.recarray.

See also

join_by

equivalent function

numpy.lib.recfunctions.recursive_fill_fields(input,output)[source]#

Fills fields from output with fields from input,with support for nested structures.

Parameters:

inputndarray

Input array.

outputndarray

Output array.

Notes

• output should be at least the same size asinput

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>a=np.array([(1,10.),(2,20.)],dtype=[('A',np.int64),('B',np.float64)])>>>b=np.zeros((3,),dtype=a.dtype)>>>rfn.recursive_fill_fields(a,b)array([(1, 10.), (2, 20.), (0,  0.)], dtype=[('A', '<i8'), ('B', '<f8')])

Go BackOpen In Tab

numpy.lib.recfunctions.rename_fields(base,namemapper)[source]#

Rename the fields from a flexible-datatype ndarray or recarray.

Nested fields are supported.

Parameters:

basendarray

Input array whose fields must be modified.

namemapperdictionary

Dictionary mapping old field names to their new version.

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>a=np.array([(1,(2,[3.0,30.])),(4,(5,[6.0,60.]))],...dtype=[('a',int),('b',[('ba',float),('bb',(float,2))])])>>>rfn.rename_fields(a,{'a':'A','bb':'BB'})array([(1, (2., [ 3., 30.])), (4, (5., [ 6., 60.]))],      dtype=[('A', '<i8'), ('b', [('ba', '<f8'), ('BB', '<f8', (2,))])])

Go BackOpen In Tab

numpy.lib.recfunctions.repack_fields(a,align=False,recurse=False)[source]#

Re-pack the fields of a structured array or dtype in memory.

The memory layout of structured datatypes allows fields at arbitrarybyte offsets. This means the fields can be separated by padding bytes,their offsets can be non-monotonically increasing, and they can overlap.

This method removes any overlaps and reorders the fields in memory so theyhave increasing byte offsets, and adds or removes padding bytes dependingon thealign option, which behaves like thealign option tonumpy.dtype.

Ifalign=False, this method produces a “packed” memory layout in whicheach field starts at the byte the previous field ended, and any paddingbytes are removed.

Ifalign=True, this methods produces an “aligned” memory layout in whicheach field’s offset is a multiple of its alignment, and the total itemsizeis a multiple of the largest alignment, by adding padding bytes as needed.

Parameters:

andarray or dtype

array or dtype for which to repack the fields.

alignboolean

If true, use an “aligned” memory layout, otherwise use a “packed” layout.

recurseboolean

If True, also repack nested structures.

Returns:

repackedndarray or dtype

Copy ofa with fields repacked, ora itself if no repacking wasneeded.

Examples

Try it in your browser!

>>>importnumpyasnp

>>>fromnumpy.libimportrecfunctionsasrfn>>>defprint_offsets(d):...print("offsets:",[d.fields[name][1]fornameind.names])...print("itemsize:",d.itemsize)...>>>dt=np.dtype('u1, <i8, <f8',align=True)>>>dtdtype({'names': ['f0', 'f1', 'f2'], 'formats': ['u1', '<i8', '<f8'], 'offsets': [0, 8, 16], 'itemsize': 24}, align=True)>>>print_offsets(dt)offsets: [0, 8, 16]itemsize: 24>>>packed_dt=rfn.repack_fields(dt)>>>packed_dtdtype([('f0', 'u1'), ('f1', '<i8'), ('f2', '<f8')])>>>print_offsets(packed_dt)offsets: [0, 1, 9]itemsize: 17

Go BackOpen In Tab

numpy.lib.recfunctions.require_fields(array,required_dtype)[source]#

Casts a structured array to a new dtype using assignment by field-name.

This function assigns from the old to the new array by name, so thevalue of a field in the output array is the value of the field with thesame name in the source array. This has the effect of creating a newndarray containing only the fields “required” by the required_dtype.

If a field name in the required_dtype does not exist in theinput array, that field is created and set to 0 in the output array.

Parameters:

andarray

array to cast

required_dtypedtype

datatype for output array

Returns:

outndarray

array with the new dtype, with field values copied from the fields inthe input array with the same name

Examples

Try it in your browser!

>>>importnumpyasnp

>>>fromnumpy.libimportrecfunctionsasrfn>>>a=np.ones(4,dtype=[('a','i4'),('b','f8'),('c','u1')])>>>rfn.require_fields(a,[('b','f4'),('c','u1')])array([(1., 1), (1., 1), (1., 1), (1., 1)],  dtype=[('b', '<f4'), ('c', 'u1')])>>>rfn.require_fields(a,[('b','f4'),('newf','u1')])array([(1., 0), (1., 0), (1., 0), (1., 0)],  dtype=[('b', '<f4'), ('newf', 'u1')])

Go BackOpen In Tab

numpy.lib.recfunctions.stack_arrays(arrays,defaults=None,usemask=True,asrecarray=False,autoconvert=False)[source]#

Superposes arrays fields by fields

Parameters:

arraysarray or sequence

Sequence of input arrays.

defaultsdictionary, optional

Dictionary mapping field names to the corresponding default values.

usemask{True, False}, optional

Whether to return a MaskedArray (or MaskedRecords isasrecarray==True) or a ndarray.

asrecarray{False, True}, optional

Whether to return a recarray (or MaskedRecords ifusemask==True)or just a flexible-type ndarray.

autoconvert{False, True}, optional

Whether automatically cast the type of the field to the maximum.

Examples

Try it in your browser!

>>>importnumpyasnp>>>fromnumpy.libimportrecfunctionsasrfn>>>x=np.array([1,2,])>>>rfn.stack_arrays(x)isxTrue>>>z=np.array([('A',1),('B',2)],dtype=[('A','|S3'),('B',float)])>>>zz=np.array([('a',10.,100.),('b',20.,200.),('c',30.,300.)],...dtype=[('A','|S3'),('B',np.double),('C',np.double)])>>>test=rfn.stack_arrays((z,zz))>>>testmasked_array(data=[(b'A', 1.0, --), (b'B', 2.0, --), (b'a', 10.0, 100.0),                   (b'b', 20.0, 200.0), (b'c', 30.0, 300.0)],             mask=[(False, False,  True), (False, False,  True),                   (False, False, False), (False, False, False),                   (False, False, False)],       fill_value=(b'N/A', 1e+20, 1e+20),            dtype=[('A', 'S3'), ('B', '<f8'), ('C', '<f8')])

Go BackOpen In Tab

numpy.lib.recfunctions.structured_to_unstructured(arr,dtype=None,copy=False,casting='unsafe')[source]#

Converts an n-D structured array into an (n+1)-D unstructured array.

The new array will have a new last dimension equal in size to thenumber of field-elements of the input array. If not supplied, the outputdatatype is determined from the numpy type promotion rules applied to allthe field datatypes.

Nested fields, as well as each element of any subarray fields, all countas a single field-elements.

Parameters:

arrndarray

Structured array or dtype to convert. Cannot contain object datatype.

dtypedtype, optional

The dtype of the output unstructured array.

copybool, optional

If true, always return a copy. If false, a view is returned ifpossible, such as when thedtype and strides of the fields aresuitable and the array subtype is one ofnumpy.ndarray,numpy.recarray ornumpy.memmap.

Changed in version 1.25.0:A view can now be returned if the fields are separated by auniform stride.

casting{‘no’, ‘equiv’, ‘safe’, ‘same_kind’, ‘unsafe’}, optional

See casting argument ofnumpy.ndarray.astype. Controls what kind ofdata casting may occur.

Returns:

unstructuredndarray

Unstructured array with one more dimension.

Examples

Try it in your browser!

>>>importnumpyasnp

>>>fromnumpy.libimportrecfunctionsasrfn>>>a=np.zeros(4,dtype=[('a','i4'),('b','f4,u2'),('c','f4',2)])>>>aarray([(0, (0., 0), [0., 0.]), (0, (0., 0), [0., 0.]),       (0, (0., 0), [0., 0.]), (0, (0., 0), [0., 0.])],      dtype=[('a', '<i4'), ('b', [('f0', '<f4'), ('f1', '<u2')]), ('c', '<f4', (2,))])>>>rfn.structured_to_unstructured(a)array([[0., 0., 0., 0., 0.],       [0., 0., 0., 0., 0.],       [0., 0., 0., 0., 0.],       [0., 0., 0., 0., 0.]])

>>>b=np.array([(1,2,5),(4,5,7),(7,8,11),(10,11,12)],...dtype=[('x','i4'),('y','f4'),('z','f8')])>>>np.mean(rfn.structured_to_unstructured(b[['x','z']]),axis=-1)array([ 3. ,  5.5,  9. , 11. ])

Go BackOpen In Tab

numpy.lib.recfunctions.unstructured_to_structured(arr,dtype=None,names=None,align=False,copy=False,casting='unsafe')[source]#

Converts an n-D unstructured array into an (n-1)-D structured array.

The last dimension of the input array is converted into a structure, withnumber of field-elements equal to the size of the last dimension of theinput array. By default all output fields have the input array’s dtype, butan output structured dtype with an equal number of fields-elements can besupplied instead.

Nested fields, as well as each element of any subarray fields, all counttowards the number of field-elements.

Parameters:

arrndarray

Unstructured array or dtype to convert.

dtypedtype, optional

The structured dtype of the output array

nameslist of strings, optional

If dtype is not supplied, this specifies the field names for the outputdtype, in order. The field dtypes will be the same as the input array.

alignboolean, optional

Whether to create an aligned memory layout.

copybool, optional

See copy argument tonumpy.ndarray.astype. If true, always return acopy. If false, anddtype requirements are satisfied, a view isreturned.

casting{‘no’, ‘equiv’, ‘safe’, ‘same_kind’, ‘unsafe’}, optional

See casting argument ofnumpy.ndarray.astype. Controls what kind ofdata casting may occur.

Returns:

structuredndarray

Structured array with fewer dimensions.

Examples

Try it in your browser!

>>>importnumpyasnp

>>>fromnumpy.libimportrecfunctionsasrfn>>>dt=np.dtype([('a','i4'),('b','f4,u2'),('c','f4',2)])>>>a=np.arange(20).reshape((4,5))>>>aarray([[ 0,  1,  2,  3,  4],       [ 5,  6,  7,  8,  9],       [10, 11, 12, 13, 14],       [15, 16, 17, 18, 19]])>>>rfn.unstructured_to_structured(a,dt)array([( 0, ( 1.,  2), [ 3.,  4.]), ( 5, ( 6.,  7), [ 8.,  9.]),       (10, (11., 12), [13., 14.]), (15, (16., 17), [18., 19.])],      dtype=[('a', '<i4'), ('b', [('f0', '<f4'), ('f1', '<u2')]), ('c', '<f4', (2,))])

Go BackOpen In Tab