NumPy 1.23.0 Release Notes#
The NumPy 1.23.0 release continues the ongoing work to improve the handling andpromotion of dtypes, increase the execution speed, clarify the documentation,and expire old deprecations. The highlights are:
Implementation of
loadtxtin C, greatly improving its performance.Exposing DLPack at the Python level for easy data exchange.
Changes to the promotion and comparisons of structured dtypes.
Improvements to f2py.
See below for the details,
New functions#
A masked array specialization of
ndenumerateis now available asnumpy.ma.ndenumerate. It provides an alternative tonumpy.ndenumerateand skips masked values by default.(gh-20020)
numpy.from_dlpackhas been added to allow easy exchange of data using theDLPack protocol. It accepts Python objects that implement the__dlpack__and__dlpack_device__methods and returns a ndarray object which isgenerally the view of the data of the input object.(gh-21145)
Deprecations#
Setting
__array_finalize__toNoneis deprecated. It must now bea method and may wish to callsuper().__array_finalize__(obj)afterchecking forNoneor if the NumPy version is sufficiently new.(gh-20766)
Using
axis=32(axis=np.MAXDIMS) in many cases had thesame meaning asaxis=None. This is deprecated andaxis=Nonemust be used instead.(gh-20920)
The hook function
PyDataMem_SetEventHookhas been deprecated and thedemonstration of its use in tool/allocation_tracking has been removed. Theability to track allocations is now built-in to python viatracemalloc.(gh-20394)
numpy.distutilshas been deprecated, as a result ofdistutilsitselfbeing deprecated. It will not be present in NumPy for Python >= 3.12, andwill be removed completely 2 years after the release of Python 3.12 For moredetails, seeStatus of numpy.distutils and migration advice.(gh-20875)
numpy.loadtxtwill now give aDeprecationWarningwhen an integerdtypeis requested but the value is formatted as a floating point number.(gh-21663)
Expired deprecations#
The
NpzFile.iteritems()andNpzFile.iterkeys()methods have beenremoved as part of the continued removal of Python 2 compatibility. Thisconcludes the deprecation from 1.15.(gh-16830)
The
alenandasscalarfunctions have been removed.(gh-20414)
The
UPDATEIFCOPYarray flag has been removed together with the enumNPY_ARRAY_UPDATEIFCOPY. The associated (and deprecated)PyArray_XDECREF_ERRwas also removed. These were all deprecated in 1.14. Theyare replaced byNPY_ARRAY_WRITEBACKIFCOPY, that requires callingPyArray_ResolveWritebackIfCopybefore the array is deallocated.(gh-20589)
Exceptions will be raised during array-like creation. When an object raisedan exception during access of the special attributes
__array__or__array_interface__, this exception was usually ignored. This behaviourwas deprecated in 1.21, and the exception will now be raised.(gh-20835)
Multidimensional indexing with non-tuple values is not allowed. Previously,code such as
arr[ind]whereind=[[0,1],[0,1]]produced aFutureWarningand was interpreted as a multidimensional index (i.e.,arr[tuple(ind)]). Now this example is treated like an array index over asingle dimension (arr[array(ind)]). Multidimensional indexing withanything but a tuple was deprecated in NumPy 1.15.(gh-21029)
Changing to a dtype of different size in F-contiguous arrays is no longerpermitted. Deprecated since Numpy 1.11.0. See below for an extendedexplanation of the effects of this change.
(gh-20722)
New Features#
crackfortran has support for operator and assignment overloading#
crackfortran parser now understands operator and assignmentdefinitions in a module. They are added in thebody list of themodule which contains a new keyimplementedby listing the namesof the subroutines or functions implementing the operator orassignment.
(gh-15006)
f2py supports reading access type attributes from derived type statements#
As a result, one does not need to usepublic orprivate statements tospecify derived type access properties.
(gh-15844)
New parameterndmin added togenfromtxt#
This parameter behaves the same asndmin fromnumpy.loadtxt.
(gh-20500)
np.loadtxt now supports quote character and single converter function#
numpy.loadtxt now supports an additionalquotechar keyword argumentwhich is not set by default. Usingquotechar='"' will read quoted fieldsas used by the Excel CSV dialect.
Further, it is now possible to pass a single callable rather than a dictionaryfor theconverters argument.
(gh-20580)
Changing to dtype of a different size now requires contiguity of only the last axis#
Previously, viewing an array with a dtype of a different item size required thatthe entire array be C-contiguous. This limitation would unnecessarily force theuser to make contiguous copies of non-contiguous arrays before being able tochange the dtype.
This change affects not onlyndarray.view, but other constructionmechanisms, including the discouraged direct assignment tondarray.dtype.
This change expires the deprecation regarding the viewing of F-contiguousarrays, described elsewhere in the release notes.
(gh-20722)
Deterministic output files for F2PY#
For F77 inputs,f2py will generatemodname-f2pywrappers.funconditionally, though these may be empty. For free-form inputs,modname-f2pywrappers.f,modname-f2pywrappers2.f90 will both be generatedunconditionally, and may be empty. This allows writing generic output rules incmake ormeson and other build systems. Older behavior can be restoredby passing--skip-empty-wrappers tof2py.Using via meson details usage.
(gh-21187)
keepdims parameter foraverage#
The parameterkeepdims was added to the functionsnumpy.averageandnumpy.ma.average. The parameter has the same meaning as itdoes in reduction functions such asnumpy.sum ornumpy.mean.
(gh-21485)
New parameterequal_nan added tonp.unique#
np.unique was changed in 1.21 to treat allNaN values as equal and returna singleNaN. Settingequal_nan=False will restore pre-1.21 behaviorto treatNaNs as unique. Defaults toTrue.
(gh-21623)
Compatibility notes#
1Dnp.linalg.norm preserves float input types, even for scalar results#
Previously, this would promote tofloat64 when theord argument wasnot one of the explicitly listed values, e.g.ord=3:
>>>f32=np.float32([1,2])>>>np.linalg.norm(f32,2).dtypedtype('float32')>>>np.linalg.norm(f32,3)dtype('float64') # numpy 1.22dtype('float32') # numpy 1.23
This change affects onlyfloat32 andfloat16 vectors withordother than-Inf,0,1,2, andInf.
(gh-17709)
Changes to structured (void) dtype promotion and comparisons#
In general, NumPy now defines correct, but slightly limited, promotion forstructured dtypes by promoting the subtypes of each field instead of raisingan exception:
>>>np.result_type(np.dtype("i,i"),np.dtype("i,d"))dtype([('f0', '<i4'), ('f1', '<f8')])
For promotion matching field names, order, and titles are enforced, howeverpadding is ignored.Promotion involving structured dtypes now always ensures native byte-order forall fields (which may change the result ofnp.concatenate)and ensures that the result will be “packed”, i.e. all fields are orderedcontiguously and padding is removed.SeeStructure comparison and promotion for further details.
Therepr of aligned structures will now never print the long form includingoffsets anditemsize unless the structure includes padding notguaranteed byalign=True.
In alignment with the above changes to the promotion logic, thecasting safety has been updated:
"equiv"enforces matching names and titles. The itemsizeis allowed to differ due to padding."safe"allows mismatching field names and titlesThe cast safety is limited by the cast safety of each includedfield.
The order of fields is used to decide cast safety of eachindividual field. Previously, the field names were used andonly unsafe casts were possible when names mismatched.
The main important change here is that name mismatches are nowconsidered “safe” casts.
(gh-19226)
NPY_RELAXED_STRIDES_CHECKING has been removed#
NumPy cannot be compiled withNPY_RELAXED_STRIDES_CHECKING=0anymore. Relaxed strides have been the default for many years andthe option was initially introduced to allow a smoother transition.
(gh-20220)
np.loadtxt has received several changes#
The row counting ofnumpy.loadtxt was fixed.loadtxt ignores fullyempty lines in the file, but counted them towardsmax_rows.Whenmax_rows is used and the file contains empty lines, these will nownot be counted. Previously, it was possible that the result contained fewerthanmax_rows rows even though more data was available to be read.If the old behaviour is required,itertools.islice may be used:
importitertoolslines=itertools.islice(open("file"),0,max_rows)result=np.loadtxt(lines,...)
While generally much faster and improved,numpy.loadtxt may now fail toconverter certain strings to numbers that were previously successfully read.The most important cases for this are:
Parsing floating point values such as
1.0into integers is now deprecated.Parsing hexadecimal floats such as
0x3p3will failAn
_was previously accepted as a thousands delimiter100_000.This will now result in an error.
If you experience these limitations, they can all be worked around by passingappropriateconverters=. NumPy now supports passing a single converterto be used for all columns to make this more convenient.For example,converters=float.fromhex can read hexadecimal float numbersandconverters=int will be able to read100_000.
Further, the error messages have been generally improved. However, this meansthat error types may differ. In particularly, aValueError is now alwaysraised when parsing of a single entry fails.
(gh-20580)
Improvements#
ndarray.__array_finalize__ is now callable#
This means subclasses can now usesuper().__array_finalize__(obj)without worrying whetherndarray is their superclass or not.The actual call remains a no-op.
(gh-20766)
Add support for VSX4/Power10#
With VSX4/Power10 enablement, the new instructions available inPower ISA 3.1 can be used to accelerate some NumPy operations,e.g., floor_divide, modulo, etc.
(gh-20821)
np.fromiter now accepts objects and subarrays#
Thenumpy.fromiter function now supports object andsubarray dtypes. Please see he function documentation forexamples.
(gh-20993)
Math C library feature detection now uses correct signatures#
Compiling is preceded by a detection phase to determine whether theunderlying libc supports certain math operations. Previously this codedid not respect the proper signatures. Fixing this enables compilationfor thewasm-ld backend (compilation for web assembly) and reducesthe number of warnings.
(gh-21154)
np.kron now maintains subclass information#
np.kron maintains subclass information now such as masked arrayswhile computing the Kronecker product of the inputs
>>>x=ma.array([[1,2],[3,4]],mask=[[0,1],[1,0]])>>>np.kron(x,x)masked_array( data=[[1, --, --, --], [--, 4, --, --], [--, --, 4, --], [--, --, --, 16]], mask=[[False, True, True, True], [ True, False, True, True], [ True, True, False, True], [ True, True, True, False]], fill_value=999999)
Warning
np.kron output now followsufunc ordering (multiply)to determine the output class type
>>>classmyarr(np.ndarray):>>>__array_priority__=-1>>>a=np.ones([2,2])>>>ma=myarray(a.shape,a.dtype,a.data)>>>type(np.kron(a,ma))==np.ndarrayFalse # Before it was True>>>type(np.kron(a,ma))==myarrTrue
(gh-21262)
Performance improvements and changes#
Fasternp.loadtxt#
numpy.loadtxt is now generally much faster than previously as most of itis now implemented in C.
(gh-20580)
Faster reduction operators#
Reduction operations likenumpy.sum,numpy.prod,numpy.add.reduce,numpy.logical_and.reduce on contiguous integer-based arrays are nowmuch faster.
(gh-21001)
Fasternp.where#
numpy.where is now much faster than previously on unpredictable/randominput data.
(gh-21130)
Faster operations on NumPy scalars#
Many operations on NumPy scalars are now significantly faster, althoughrare operations (e.g. with 0-D arrays rather than scalars) may be slowerin some cases.However, even with these improvements users who want the best performancefor their scalars, may want to convert a known NumPy scalar into a Pythonone usingscalar.item().
(gh-21188)
Fasternp.kron#
numpy.kron is about 80% faster as the product is now computedusing broadcasting.
(gh-21354)