Developing Your Own Scientific Python Code

In many cases, scientific research takes you into totally new areasof knowledge, never before explored by others. This means thecomputational work you need to do may be totally new as well. Althoughtypically such code development still happens in C or FORTRAN, Pythonis growing in popularity. This is especially true in physics.

If you are just working on a small code base, a basic text editor is fine;however,once your project reaches a certain size, using a proper IDE is a hugeadvantage. Luckily, an open-source project seeks to fillthis exact niche: Spyder. Spyder is available for Windows, Mac OS X andLinux. You should be able to find a package for your distribution, but if not,you always can grab the binaries or source code at the mainSpyder Website.

Spyder actually iswritten in Python, and it has been designed witha plugin architecture. This means you can add extra functionalityby installing plugins. If you can't find a plugin for the functionalityyou need, you always can write your own.

When you start Spyder, several panes open and atemporary Python script that you can use to start editing appears.The main pane is the editor, where the temporary script is loaded andready for you to start working. The right-hand side of the window isbroken into two more panes. The bottom pane is the console to a runningPython interpreter. Here you can see that Spyder automatically loadsNumPy, SciPy and matplotlib on startup, so you already havemost of the tools you likely will need ready to go.

Figure 1. When you start Spyder, several information panes alongwith the main editor pane appear.

You can usethis console just as you would any other Python interpreter. The toppane has multiple tabs. The first tab that opens at start up is an objectinspector. This tab lets you look at the details of any objects beingused in your code. The other tabs available are a variableinspector and a file explorer.

Figure 2. The object inspector lets you look at the details of any objects youmay want to use.

Let's begin by looking at the main editorpane. Like any other programming editor, Spyder providesfull-color highlighting of Python syntax. Rope is used to provide codeintrospection capabilities to the editor. If you start typing a functioncall, Spyder makes suggestions for code completion.

Pyflakes provideson-the-fly code analysis. Any errors in your code are highlighted rightaway with a triangle symbol in the margin. When you hover over it,details for the error are displayed in a pop-up window. You also can setbreakpoints in the editor that are used by the Python debugger. This way,you have a bit of control over how your code runs when you run it underthe pdb debugger.

Figure 3. You can set all of the options for the editor window in thepreferences section.

The console pane provides a full set of tools for controlling multipleinterpreters. When you run a script from the editor, youhave the option of starting up a new interpreter in which to run it. Or, youcan run it in an existing interpreter. You can set this behavior in Spyder'spreferences. You also simply can create a new interpreter fromthe console pane directly.

Figure 4. The preferences for the console section control things like whetherit is monitored and so forth.

Also interesting is that you can create IPythoninterpreters in the console. Then, you have all of the extra functionalityprovided by IPython at your disposal. One of these advanced featuresis the ability to use multiple IPython engines in parallel. From theconsole in Spyder, you can create these IPython engines that will runin the background, ready to be used for whatever parallel processingyou may have in mind. So, not only can you develop your new scientificcode as a parallel program, you also can work with it directly from Spyder,in parallel. All of these extra interpreters and engines run as separateprocesses, which means they will not affect Spyder itself or cause itto hang if something bad happens within one of the Python interpreters.

One of the extra tabs in the top-right pane brings up the variableexplorer. This pane gives you a list of all the variablescurrently active in the memory space within Spyder. It showsthe name, type, size and value for each of the variables accessible inthe global namespace. This applies to both the internal Python interpreterand any external ones. The variable explorer can handle all of thestandard data types, like strings, integers and floats. It also includesan array editor that can be used to edit lists and tuples. The arrayeditor provides a nice environment for editing complex data types.

Figure 5. The variable explorer gives you a list of everything within theglobal namespace.

Spyderprovides even more tools though. By right-clicking on a list or tuple,you can do some basic data analysis. Spyder allows you to plot the valuesin the data object to see what it looks like. Or, you canlook at a histogram of the values if a statistical analysis would makemore sense.

Figure 6. Spyder lets you plot the data within a list or tuple.

All of these tools are handy, but by themselves, they aren't enough if youare developing a large code base. In such cases, you will need somekind of project-level organization. Spyder can help in that situation too. Youcan create a project in order to encapsulate a group of files as a singleunit. Creating a new project makes a new folder to store all of theassociated files. Opening this new project creates a new pane whereyou can work with the project files. By right-clicking on the project,you can create a new file, folder, module or package. When you createa new file, Spyder opens it up in the editor, ready for you to start working onit.

Figure 7. If you are working on a larger code base, the project explorer willcome in very handy.

The last tool you should be aware of is the profiler. Thefirst step in program development is writing code that works. After that,your job is to write code that is as efficient as possible. The ruleof thumb is to optimize last, and only what needs to be optimized. But,what part of your program is that? Without reliable measurements, youwon't know what needs optimizing. In Spyder, you can click on the menuitem Run→Profile or press F10. This runs your code under thePython profiler, giving you a breakdown of where all the time isbeing spent. Once you have this information, you can focus your energieswhere they will do the most good.

Figure 8. The profiler opens a new pane at the bottom of the window, showingyou numbers of calls and time for each function.

Hopefully, you can take Spyder and run with it while developing yourown scientific Python code. Although lots of IDEs exist fordeveloping code, there aren't very many setups geared towarddeveloping scientific code. With Spyder, you should have a head startin developing your new breakthrough code, solving the problem that couldwin you the next Nobel Prize.