Recommended development setup¶

Since NumPy contains parts written in C and Cython that need to becompiled before use, make sure you have the necessary compilers and Pythondevelopment headers installed - seeBuilding from source.

Having compiled code also means that importing NumPy from the developmentsources needs some additional steps, which are explained below. For the restof this chapter we assume that you have set up your git repo as described inWorking with NumPy source code.

To build the development version of NumPy and run tests, spawninteractive shells with the Python import paths properly set up etc.,do one of:

$ python runtests.py -v$ python runtests.py -v -s random$ python runtests.py -v -t numpy/core/tests/test_nditer.py::test_iter_c_order$ python runtests.py --ipython$ python runtests.py --python somescript.py$ python runtests.py --bench$ python runtests.py -g -m full

This builds NumPy first, so the first time it may take a few minutes. Ifyou specify-n, the tests are run against the version of NumPy (ifany) found on current PYTHONPATH.

When specifying a target using-s,-t, or--python, additionalarguments may be forwarded to the target embedded byruntests.py by passingthe extra arguments after a bare--. For example, to run a test method withthe--pdb flag forwarded to the target, run the following:

$ python runtests.py -t numpy/tests/test_scripts.py:test_f2py -- --pdb

When using pytest as a target (the default), you canmatch test names using python operators by passing the-k argument to pytest:

$ python runtests.py -v -t numpy/core/tests/test_multiarray.py -- -k "MatMul and not vector"

Usingruntests.py is the recommended approach to running tests.There are also a number of alternatives to it, for example in-placebuild or installing to a virtualenv. See the FAQ below for details.

Building in-place¶

For development, you can set up an in-place build so that changes made to.py files have effect without rebuild. First, run:

$ python setup.py build_ext -i

This allows you to import the in-place built NumPyfrom the repo basedirectory only. If you want the in-place build to be visible outside thatbase dir, you need to point yourPYTHONPATH environment variable to thisdirectory. Some IDEs (Spyder for example) have utilities to managePYTHONPATH. On Linux and OSX, you can run the command:

$ export PYTHONPATH=$PWD

and on Windows:

$ set PYTHONPATH=/path/to/numpy

Now editing a Python source file in NumPy allows you to immediatelytest and use your changes (in.py files), by simply restarting theinterpreter.

Note that another way to do an inplace build visible outside the repo base diris withpythonsetup.pydevelop. Instead of adjustingPYTHONPATH, thisinstalls a.egg-link file into your site-packages as well as adjusts theeasy-install.pth there, so its a more permanent (and magical) operation.

Other build options¶

It’s possible to do a parallel build withnumpy.distutils with the-j option;seeParallel builds for more details.

In order to install the development version of NumPy insite-packages, usepythonsetup.pyinstall--user.

A similar approach to in-place builds and use ofPYTHONPATH but outside thesource tree is to use:

$ python setup.py install --prefix /some/owned/folder$ export PYTHONPATH=/some/owned/folder/lib/python3.4/site-packages

Using virtualenvs¶

A frequently asked question is “How do I set up a development version of NumPyin parallel to a released version that I use to do my job/research?”.

One simple way to achieve this is to install the released version insite-packages, by using a binary installer or pip for example, and setup the development version in a virtualenv. First installvirtualenv (optionally usevirtualenvwrapper), then create yourvirtualenv (named numpy-dev here) with:

$ virtualenv numpy-dev

Now, whenever you want to switch to the virtual environment, you can use thecommandsourcenumpy-dev/bin/activate, anddeactivate to exit from thevirtual environment and back to your previous shell.

Running tests¶

Besides usingruntests.py, there are various ways to run the tests. Insidethe interpreter, tests can be run like this:

>>>np.test()>>>np.test('full')# Also run tests marked as slow>>>np.test('full',verbose=2)# Additionally print test name/file

Or a similar way from the command line:

$ python -c "import numpy as np; np.test()"

Tests can also be run withpytestnumpy, however then the NumPy-specificplugin is not found which causes strange side effects

Running individual test files can be useful; it’s much faster than running thewhole test suite or that of a whole module (example:np.random.test()).This can be done with:

$ python path_to_testfile/test_file.py

That also takes extra arguments, like--pdb which drops you into the Pythondebugger when a test fails or an exception is raised.

Running tests withtox is also supported. For example, to build NumPy andrun the test suite with Python 3.4, use:

$ tox -e py34

For more extensive information, seeTesting Guidelines

Note: do not run the tests from the root directory of your numpy git repo without ``runtests.py``,that will result in strange test errors.

Rebuilding & cleaning the workspace¶

Rebuilding NumPy after making changes to compiled code can be done with thesame build command as you used previously - only the changed files will bere-built. Doing a full build, which sometimes is necessary, requires cleaningthe workspace first. The standard way of doing this is (note: deletes anyuncommitted files!):

$ git clean -xdf

When you want to discard all changes and go back to the last commit in therepo, use one of:

$ git checkout .$ git reset --hard

Debugging¶

Another frequently asked question is “How do I debug C code inside NumPy?”.The easiest way to do this is to first write a Python script that invokes the Ccode whose execution you want to debug. For instancemytest.py:

fromnumpyimportlinspacex=np.arange(5)np.empty_like(x)

Now, you can run:

$ gdb --args python runtests.py -g --python mytest.py

And then in the debugger:

(gdb)breakarray_empty_like(gdb)run

The execution will now stop at the corresponding C function and you can stepthrough it as usual. With the Python extensions for gdb installed (often thedefault on Linux), a number of useful Python-specific commands are available.For example to see where in the Python code you are, usepy-list. For moredetails, seeDebuggingWithGdb.

Instead of plaingdb you can of course use your favouritealternative debugger; run it on the python binary with argumentsruntests.py-g--pythonmytest.py.

Building NumPy with a Python built with debug support (on Linux distributionstypically packaged aspython-dbg) is highly recommended.

Understanding the code & getting started¶

The best strategy to better understand the code base is to pick something youwant to change and start reading the code to figure out how it works. When indoubt, you can ask questions on the mailing list. It is perfectly okay if yourpull requests aren’t perfect, the community is always happy to help. As avolunteer project, things do sometimes get dropped and it’s totally fine toping us if something has sat without a response for about two to four weeks.

So go ahead and pick something that annoys or confuses you about numpy,experiment with the code, hang around for discussions or go through thereference documents to try to fix it. Things will fall in place and soonyou’ll have a pretty good understanding of the project as a whole. Good Luck!