Compiled testing modules renamed and made private#

The following compiled modules have been renamed and made private:

• umath_tests ->_umath_tests

• test_rational ->_rational_tests

• multiarray_tests ->_multiarray_tests

• struct_ufunc_test ->_struct_ufunc_tests

• operand_flag_tests ->_operand_flag_tests

Theumath_tests module is still available for backwards compatibility, butwill be removed in the future.

TheNpzFile returned bynp.savez is now acollections.abc.Mapping#

This means it behaves like a readonly dictionary, and has a new.values()method andlen() implementation.

For python 3, this means that.iteritems(),.iterkeys() have beendeprecated, and.keys() and.items() now return views and not lists.This is consistent with how the builtindict type changed between python 2and python 3.

Under certain conditions,nditer must be used in a context manager#

When using annumpy.nditer with the"writeonly" or"readwrite" flags, thereare some circumstances where nditer doesn’t actually give you a view of thewritable array. Instead, it gives you a copy, and if you make changes to thecopy, nditer later writes those changes back into your actual array. Currently,this writeback occurs when the array objects are garbage collected, which makesthis API error-prone on CPython and entirely broken on PyPy. Therefore,nditer should now be used as a context manager whenever it is usedwith writeable arrays, e.g.,withnp.nditer(...)asit:.... You may alsoexplicitly callit.close() for cases where a context manager is unusable,for instance in generator expressions.

Numpy has switched to using pytest instead of nose for testing#

The last nose release was 1.3.7 in June, 2015, and development of that tool hasended, consequently NumPy has now switched to using pytest. The old decoratorsand nose tools that were previously used by some downstream projects remainavailable, but will not be maintained. The standard testing utilities,assert_almost_equal and such, are not be affected by this change except forthe nose specific functionsimport_nose andraises. Those functions arenot used in numpy, but are kept for downstream compatibility.

Numpy no longer monkey-patchesctypes with__array_interface__#

Previously numpy added__array_interface__ attributes to all the integertypes fromctypes.

np.ma.notmasked_contiguous andnp.ma.flatnotmasked_contiguous always return lists#

This is the documented behavior, but previously the result could be any ofslice, None, or list.

All downstream users seem to check for theNone result fromflatnotmasked_contiguous and replace it with[]. Those callers willcontinue to work as before.

np.squeeze restores old behavior of objects that cannot handle anaxis argument#

Prior to version1.7.0,numpy.squeeze did not have anaxis argument andall empty axes were removed by default. The incorporation of anaxisargument made it possible to selectively squeeze single or multiple empty axes,but the old API expectation was not respected because axes could still beselectively removed (silent success) from an object expecting all empty axes tobe removed. That silent, selective removal of empty axes for objects expectingthe old behavior has been fixed and the old behavior restored.

unstructured void array’s.item method now returns a bytes object#

.item now returns abytes object instead of a buffer or byte array.This may affect code which assumed the return value was mutable, which is nolonger the case.

copy.copy andcopy.deepcopy no longer turnmasked into an array#

Sincenp.ma.masked is a readonly scalar, copying should be a no-op. Thesefunctions now behave consistently withnp.copy().

Multifield Indexing of Structured Arrays will still return a copy#

The change that multi-field indexing of structured arrays returns a viewinstead of a copy is pushed back to 1.16. A new methodnumpy.lib.recfunctions.repack_fields has been introduced to help mitigatethe effects of this change, which can be used to write code compatible withboth numpy 1.15 and 1.16. For more information on how to update code to accountfor this future change see the “accessing multiple fields” section of theuser guide.

New functionsnpy_get_floatstatus_barrier andnpy_clear_floatstatus_barrier#

Functionsnpy_get_floatstatus_barrier andnpy_clear_floatstatus_barrierhave been added and should be used in place of thenpy_get_floatstatus``and``npy_clear_status functions. Optimizing compilers like GCC 8.1 and Clangwere rearranging the order of operations when the previous functions were usedin the ufunc SIMD functions, resulting in the floatstatus flags being checkedbefore the operation whose status we wanted to check was run. See#10339.

Changes toPyArray_GetDTypeTransferFunction#

PyArray_GetDTypeTransferFunction now defaults to using user-definedcopyswapn /copyswap for user-defined dtypes. If this causes asignificant performance hit, consider implementingcopyswapn to reflect theimplementation ofPyArray_GetStridedCopyFn. See#10898.

np.gcd andnp.lcm ufuncs added for integer and objects types#

These compute the greatest common divisor, and lowest common multiple,respectively. These work on all the numpy integer types, as well as thebuiltin arbitrary-precisionDecimal andlong types.

Support for cross-platform builds for iOS#

The build system has been modified to add support for the_PYTHON_HOST_PLATFORM environment variable, used bydistutils whencompiling on one platform for another platform. This makes it possible tocompile NumPy for iOS targets.

This only enables you to compile NumPy for one specific platform at a time.Creating a full iOS-compatible NumPy package requires building for the 5architectures supported by iOS (i386, x86_64, armv7, armv7s and arm64), andcombining these 5 compiled builds products into a single “fat” binary.

return_indices keyword added fornp.intersect1d#

New keywordreturn_indices returns the indices of the two input arraysthat correspond to the common elements.

np.quantile andnp.nanquantile#

Likenp.percentile andnp.nanpercentile, but takes quantiles in [0, 1]rather than percentiles in [0, 100].np.percentile is now a thin wrapperaroundnp.quantile with the extra step of dividing by 100.

Build system#

Added experimental support for the 64-bit RISC-V architecture.

np.einsum updates#

Syncs einsum path optimization tech betweennumpy andopt_einsum. Inparticular, thegreedy path has received many enhancements by @jcmgray. Afull list of issues fixed are:

• Arbitrary memory can be passed into thegreedy path. Fixes gh-11210.

• The greedy path has been updated to contain more dynamic programming ideaspreventing a large number of duplicate (and expensive) calls that figure outthe actual pair contraction that takes place. Now takes a few seconds onseveral hundred input tensors. Useful for matrix product state theories.

• Reworks the broadcasting dot error catching found in gh-11218 gh-10352 to bea bit earlier in the process.

• Enhances thecan_dot functionality that previous missed an edge case (partof gh-11308).

np.ufunc.reduce and related functions now accept an initial value#

np.ufunc.reduce,np.sum,np.prod,np.min andnp.max allnow accept aninitial keyword argument that specifies the value to startthe reduction with.

np.flip can operate over multiple axes#

np.flip now accepts None, or tuples of int, in itsaxis argument. Ifaxis is None, it will flip over all the axes.

histogram andhistogramdd functions have moved tonp.lib.histograms#

These were originally found innp.lib.function_base. They are stillavailable under their un-scopednp.histogram(dd) names, andto maintain compatibility, aliased atnp.lib.function_base.histogram(dd).

Code that doesfromnp.lib.function_baseimport* will need to be updatedwith the new location, and should consider not usingimport* in future.

histogram will accept NaN values when explicit bins are given#

Previously it would fail when trying to compute a finite range for the data.Since the range is ignored anyway when the bins are given explicitly, this errorwas needless.

Note that callinghistogram on NaN values continues to raise theRuntimeWarning s typical of working with nan values, which can be silencedas usual witherrstate.

histogram works on datetime types, when explicit bin edges are given#

Dates, times, and timedeltas can now be histogrammed. The bin edges must bepassed explicitly, and are not yet computed automatically.

histogram “auto” estimator handles limited variance better#

No longer does an IQR of 0 result inn_bins=1, rather the number of binschosen is related to the data size in this situation.

The edges returned byhistogram` andhistogramdd now match the data float type#

When passednp.float16,np.float32, ornp.longdouble data, thereturned edges are now of the same dtype. Previously,histogram would onlyreturn the same type if explicit bins were given, andhistogram wouldproducefloat64 bins no matter what the inputs.

histogramdd allows explicit ranges to be given in a subset of axes#

Therange argument ofnumpy.histogramdd can now containNone values toindicate that the range for the corresponding axis should be computed from thedata. Previously, this could not be specified on a per-axis basis.

The normed arguments ofhistogramdd andhistogram2d have been renamed#

These arguments are now calleddensity, which is consistent withhistogram. The old argument continues to work, but the new name should bepreferred.

np.r_ works with 0d arrays, andnp.ma.mr_ works withnp.ma.masked#

0d arrays passed to ther_ andmr_ concatenation helpers are now treated asthough they are arrays of length 1. Previously, passing these was an error.As a result,numpy.ma.mr_ now works correctly on themasked constant.

np.ptp accepts akeepdims argument, and extended axis tuples#

np.ptp (peak-to-peak) can now work over multiple axes, just likenp.maxandnp.min.

MaskedArray.astype now is identical tondarray.astype#

This means it takes all the same arguments, making more code written forndarray work for masked array too.

Enable AVX2/AVX512 at compile time#

Change to simd.inc.src to allow use of AVX2 or AVX512 at compile time. Previouslycompilation for avx2 (or 512) with -march=native would still use the SSEcode for the simd functions even when the rest of the code got AVX2.

nan_to_num always returns scalars when receiving scalar or 0d inputs#

Previously an array was returned for integer scalar inputs, which isinconsistent with the behavior for float inputs, and that of ufuncs in general.For all types of scalar or 0d input, the result is now a scalar.

np.flatnonzero works on numpy-convertible types#

np.flatnonzero now usesnp.ravel(a) instead ofa.ravel(), so itworks for lists, tuples, etc.

np.interp returns numpy scalars rather than builtin scalars#

Previouslynp.interp(0.5,[0,1],[10,20]) would return afloat, butnow it returns anp.float64 object, which more closely matches the behaviorof other functions.

Additionally, the special case ofnp.interp(object_array_0d,...) is nolonger supported, asnp.interp(object_array_nd) was never supported anyway.

As a result of this change, theperiod argument can now be used on 0darrays.

Allow dtype field names to be unicode in Python 2#

Previouslynp.dtype([(u'name',float)]) would raise aTypeError inPython 2, as only bytestrings were allowed in field names. Now any unicodestring field names will be encoded with theascii codec, raising aUnicodeEncodeError upon failure.

This change makes it easier to write Python 2/3 compatible code usingfrom__future__importunicode_literals, which previously would causestring literal field names to raise a TypeError in Python 2.

Comparison ufuncs acceptdtype=object, overriding the defaultbool#

This allows object arrays of symbolic types, which override== and otheroperators to return expressions, to be compared elementwise withnp.equal(a,b,dtype=object).

sort functions acceptkind='stable'#

Up until now, to perform a stable sort on the data, the user must do:

>>>np.sort([5,2,6,2,1],kind='mergesort')[1, 2, 2, 5, 6]

because merge sort is the only stable sorting algorithm available inNumPy. However, having kind=’mergesort’ does not make it explicit thatthe user wants to perform a stable sort thus harming the readability.

This change allows the user to specify kind=’stable’ thus clarifyingthe intent.

Do not make temporary copies for in-place accumulation#

When ufuncs perform accumulation they no longer make temporary copies becauseof the overlap between input an output, that is, the next element accumulatedis added before the accumulated result is stored in its place, hence theoverlap is safe. Avoiding the copy results in faster execution.

linalg.matrix_power can now handle stacks of matrices#

Like other functions inlinalg,matrix_power can now deal with arraysof dimension larger than 2, which are treated as stacks of matrices. As partof the change, to further improve consistency, the name of the first argumenthas been changed toa (fromM), and the exceptions for non-squarematrices have been changed toLinAlgError (fromValueError).

Increased performance inrandom.permutation for multidimensional arrays#

permutation uses the fast path inrandom.shuffle for all inputarray dimensions. Previously the fast path was only used for 1-d arrays.

Generalized ufuncs now acceptaxes,axis andkeepdims arguments#

One can control over which axes a generalized ufunc operates by passing in anaxes argument, a list of tuples with indices of particular axes. Forinstance, for a signature of(i,j),(j,k)->(i,k) appropriate for matrixmultiplication, the base elements are two-dimensional matrices and these aretaken to be stored in the two last axes of each argument. The correspondingaxes keyword would be[(-2,-1),(-2,-1),(-2,-1)]. If one wanted touse leading dimensions instead, one would pass in[(0,1),(0,1),(0,1)].

For simplicity, for generalized ufuncs that operate on 1-dimensional arrays(vectors), a single integer is accepted instead of a single-element tuple, andfor generalized ufuncs for which all outputs are scalars, the (empty) outputtuples can be omitted. Hence, for a signature of(i),(i)->() appropriatefor an inner product, one could pass inaxes=[0,0] to indicate that thevectors are stored in the first dimensions of the two inputs arguments.

As a short-cut for generalized ufuncs that are similar to reductions, i.e.,that act on a single, shared core dimension such as the inner product exampleabove, one can pass anaxis argument. This is equivalent to passing inaxes with identical entries for all arguments with that core dimension(e.g., for the example above,axes=[(axis,),(axis,)]).

Furthermore, like for reductions, for generalized ufuncs that have inputs thatall have the same number of core dimensions and outputs with no core dimension,one can pass inkeepdims to leave a dimension with size 1 in the outputs,thus allowing proper broadcasting against the original inputs. The location ofthe extra dimension can be controlled withaxes. For instance, for theinner-product example,keepdims=True,axes=[-2,-2,-2] would act on theinner-product example,keepdims=True,axis=-2 would act on theone-but-last dimension of the input arguments, and leave a size 1 dimension inthat place in the output.

float128 values now print correctly on ppc systems#

Previously printing float128 values was buggy on ppc, since the specialdouble-double floating-point-format on these systems was not accounted for.float128s now print with correct rounding and uniqueness.

Warning to ppc users: You should upgrade glibc if it is version <=2.23,especially if using float128. On ppc, glibc’s malloc in these version oftenmisaligns allocated memory which can crash numpy when using float128 values.

Newnp.take_along_axis andnp.put_along_axis functions#

When used on multidimensional arrays,argsort,argmin,argmax, andargpartition return arrays that are difficult to use as indices.take_along_axis provides an easy way to use these indices to lookup valueswithin an array, so that:

np.take_along_axis(a,np.argsort(a,axis=axis),axis=axis)

is the same as:

np.sort(a,axis=axis)

np.put_along_axis acts as the dual operation for writing to these indiceswithin an array.