IPython Sphinx Directive

Note

The IPython Sphinx Directive is in ‘beta’ and currently underactive development. Improvements to the code or documentation are welcome!

Theipython directive is a stateful shell that can be usedin reStructured text files.

It knows about standard ipython prompts, and extracts the input and outputlines. These prompts will be renumbered starting at1. The inputs will befed to an embedded ipython interpreter and the outputs from that interpreterwill be inserted as well. For example, code blocks like the following:

..ipython::In[136]:x=2In[137]:x**3Out[137]:8

will be rendered as

In [1]:x=2In [2]:x**3Out[2]:8

Note

This tutorial should be read side-by-side with the Sphinx sourcefor this document because otherwise you will see only the renderedoutput and not the code that generated it. Excepting the exampleabove, we will not in general be showing the literal ReST in thisdocument that generates the rendered output.

Directive and options

The IPython directive takes a number of options detailed here.

..ipython::

Create an IPython directive.

:doctest:

Run a doctest on IPython code blocks in rst.

:python:

Used to indicate that the relevant code block does not have IPython prompts.

:okexcept:

Allow the code block to raise an exception.

:okwarning:

Allow the code block to emit an warning.

:suppress:

Silence any warnings or expected errors.

:verbatim:

A noop that allows for any text to be syntax highlighted as valid IPython code.

:savefig:OUTFILE[IMAGE_OPTIONS]

Save output from matplotlib tooutfile.

It’s important to note that all of these options can be used for the entiredirective block or they can decorate individual lines of code as explainedinPseudo-Decorators.

Persisting the Python session across IPython directive blocks

The state from previous sessions is stored, and standard error istrapped. At doc build time, ipython’s output and std err will beinserted, and prompts will be renumbered. So the prompt below shouldbe renumbered in the rendered docs, and pick up where the block aboveleft off.

In [3]:z=x*3# x is recalled from previous blockIn [4]:zOut[4]:6In [5]:print(z)6In [6]:q=z[)# this is a syntax error -- we trap ipy exceptions------------------------------------------------------------File"<ipython console>",line1q=z[)# this is a syntax error -- we trap ipy exceptions^SyntaxError: invalid syntax

Adding documentation tests to your IPython directive

The embedded interpreter supports some limited markup. For example,you can put comments in your ipython sessions, which are reportedverbatim. There are some handy “pseudo-decorators” that let youdoctest the output. The inputs are fed to an embedded ipythonsession and the outputs from the ipython session are inserted intoyour doc. If the output in your doc and in the ipython session don’tmatch on a doctest assertion, an error will occur.

In [7]:x='hello world'# this will raise an error if the ipython output is differentIn [8]:x.upper()Out[8]:'HELLO WORLD'# some readline features cannot be supported, so we allow# "verbatim" blocks, which are dumped in verbatim except prompts# are continuously numberedIn [9]:x.st<TAB>x.startswith  x.strip

For more information on @doctest decorator, please refer to the end of this page in Pseudo-Decorators section.

Multi-line input

Multi-line input is supported.

In [10]:url='http://ichart.finance.yahoo.com/table.csv?s=CROX\   ....:&d=9&e=22&f=2009&g=d&a=1&br=8&c=2006&ignore=.csv'   ....:In [11]:print(url.split('&'))['http://ichart.finance.yahoo.com/table.csv?s=CROX', 'd=9', 'e=22',

Testing directive outputs

The IPython Sphinx Directive makes it possible to test the outputs that you provide with your code. To do this,decorate the contents in your directive block with one of the options listedabove.

If an IPython doctest decorator is found, it will take these steps when your documentation is built:

1. Run theinput lines in your IPython directive block against the current Python kernel (remember that the sessionpersists across IPython directive blocks);

2. Compare theoutput of this with the output text that you’ve put in the IPython directive block (what comesafterOut[NN]);

  1. If there is a difference, the directive will raise an error and your documentation build will fail.

You can do doctesting on multi-line output as well. Just be carefulwhen using non-deterministic inputs like random numbers in the ipythondirective, because your inputs are run through a live interpreter, soif you are doctesting random output you will get an error. Here we“seed” the random number generator for deterministic output, and wesuppress the seed line so it doesn’t show up in the rendered output

In [12]:importnumpy.randomIn [13]:numpy.random.rand(10,2)Out[13]:array([[0.64524308, 0.59943846],       [0.47102322, 0.8715456 ],       [0.29370834, 0.74776844],       [0.99539577, 0.1313423 ],       [0.16250302, 0.21103583],       [0.81626524, 0.1312433 ],       [0.67338089, 0.72302393],       [0.7566368 , 0.07033696],       [0.22591016, 0.77731835],       [0.0072729 , 0.34273127]])

For more information on @suppress and @doctest decorators, please refer to the end of this file inPseudo-Decorators section.

Another demonstration of multi-line input and output

In [14]:print(x)jdhIn [15]:foriinrange(10):   ....:print(i)   ....:   ....:0123456789

Most of the “pseudo-decorators” can be used an options to ipythonmode. For example, to setup matplotlib pylab but suppress the output,you can do. When using the matplotlibuse directive, it shouldoccur before any import of pylab. This will not show up in therendered docs, but the commands will be executed in the embeddedinterpreter and subsequent line numbers will be incremented to reflectthe inputs:

..ipython:::suppress::okexcept:In[144]:frommatplotlib.pylabimport*In[145]:ion()

Likewise, you can set:doctest: or:verbatim: to apply thesesettings to the entire block. For example,

In [16]:cdmpl/examples//home/jdhunter/mpl/examplesIn [17]:pwdOut[17]:'/home/jdhunter/mpl/examples'In [18]:cdmpl/examples/<TAB>mpl/examples/animation/        mpl/examples/misc/mpl/examples/api/              mpl/examples/mplot3d/mpl/examples/axes_grid/        mpl/examples/pylab_examples/mpl/examples/event_handling/   mpl/examples/widgetsIn [19]:cdmpl/examples/widgets//home/msierig/mpl/examples/widgetsIn [20]:!wc*    2    12    77 README.txt   40    97   884 buttons.py   26    90   712 check_buttons.py   19    52   416 cursor.py  180   404  4882 menu.py   16    45   337 multicursor.py   36   106   916 radio_buttons.py   48   226  2082 rectangle_selector.py   43   118  1063 slider_demo.py   40   124  1088 span_selector.py  450  1274 12457 total

You can create one or more pyplot plots and insert them with the@savefig decorator.

For more information on @savefig decorator, please refer to the end of this page in Pseudo-Decorators section.

In [21]:plot([1,2,3]);# use a semicolon to suppress the outputIn [22]:hist(np.random.randn(10000),100);
_images/plot_simple.png_images/hist_simple.png

In a subsequent session, we can update the current figure with sometext, and then resave

In [23]:ylabel('number')Out[23]:Text(38.347222222222214, 0.5, 'number')In [24]:title('normal distribution')Out[24]:Text(0.5, 1.0, 'normal distribution')In [25]:grid(True)
_images/hist_with_text.png

You can also have function definitions included in the source.

In [26]:defsquare(x):   ....:"""   ....:    An overcomplicated square function as an example.   ....:    """   ....:ifx<0:   ....:x=abs(x)   ....:y=x*x   ....:returny   ....:

Then call it from a subsequent section.

In [27]:square(3)Out[27]:9In [28]:square(-2)Out[28]:4

Writing Pure Python Code

Pure python code is supported by the optional argumentpython. In this purepython syntax you do not include the output from the python interpreter. Thefollowing markup:

..ipython::pythonfoo='bar'print(foo)foo=2foo**2

Renders as

In [29]:foo='bar'In [30]:print(foo)barIn [31]:foo=2In [32]:foo**2Out[32]:4

We can even plot from python, using the savefig decorator, as well as, suppressoutput with a semicolon

In [33]:plot([1,2,3]);
_images/plot_simple_python.png

For more information on @savefig decorator, please refer to the end of this page in Pseudo-Decorators section.

Similarly, std err is inserted

In [34]:foo='bar'In [35]:foo[)  Cell In[35], line 1    foo[)        ^SyntaxError: closing parenthesis ')' does not match opening parenthesis '['

Handling Comments

Comments are handled and state is preserved

# comments are handledIn [36]:print(foo)bar

If you don’t see the next code block then the options work.

Splitting Python statements across lines

Multi-line input is handled.

In [37]:line='Multi\   ....:        line &\   ....:        support &\   ....:        works'   ....:In [38]:print(line.split('&'))['Multi        line ', '        support ', '        works']

Functions definitions are correctly parsed

In [39]:defsquare(x):   ....:"""   ....:    An overcomplicated square function as an example.   ....:    """   ....:ifx<0:   ....:x=abs(x)   ....:y=x*x   ....:returny   ....:

And persist across sessions

In [40]:print(square(3))9In [41]:print(square(-2))4

Pretty much anything you can do with the ipython code, you can do witha simple python script. Obviously, though it doesn’t make senseto use the doctest option.

Pseudo-Decorators

Here are the supported decorators, and any optional arguments theytake. Some of the decorators can be used as options to the entireblock (egverbatim andsuppress), and some only apply to theline just below them (egsavefig).

@suppress

execute the ipython input block, but suppress the input and outputblock from the rendered output. Also, can be applied to the entire..ipython block as a directive option with:suppress:.

@verbatim

insert the input and output block in verbatim, but auto-incrementthe line numbers. Internally, the interpreter will be fed an emptystring, so it is a no-op that keeps line numbering consistent.Also, can be applied to the entire..ipython block as adirective option with:verbatim:.

@savefig OUTFILE [IMAGE_OPTIONS]

save the figure to the static directory and insert it into thedocument, possibly binding it into a minipage and/or puttingcode/figure label/references to associate the code and thefigure. Takes args to pass to the image directive (scale,width, etc can be kwargs); seeimage optionsfor details.

@doctest

Compare the pasted in output in the ipython block with the outputgenerated at doc build time, and raise errors if they don’tmatch. Also, can be applied to the entire..ipython block as adirective option with:doctest:.

Configuration Options

ipython_savefig_dir

The directory in which to save the figures. This is relative to theSphinx source directory. The default ishtml_static_path.

ipython_rgxin

The compiled regular expression to denote the start of IPython inputlines. The default isre.compile('In[(d+)]:s?(.*)s*'). Youshouldn’t need to change this.

ipython_rgxout

The compiled regular expression to denote the start of IPython outputlines. The default isre.compile('Out[(d+)]:s?(.*)s*'). Youshouldn’t need to change this.

ipython_promptin

The string to represent the IPython input prompt in the generated ReST.The default is'In[%d]:'. This expects that the line numbers are usedin the prompt.

ipython_promptout

The string to represent the IPython prompt in the generated ReST. Thedefault is'Out[%d]:'. This expects that the line numbers are usedin the prompt.

Automatically generated documentation

Sphinx directive to support embedded IPython code.

IPython provides an extension forSphinx tohighlight and run code.

This directive allows pasting of entire interactive IPython sessions, promptsand all, and their code will actually get re-executed at doc build time, withall prompts renumbered sequentially. It also allows you to input code as a purepython input by giving the argument python to the directive. The output lookslike an interactive ipython section.

Here is an example of how the IPython directive canrun python code, at build time.

In [1]:1+1Out[1]:2In [2]:importdatetime   ...:datetime.date.fromisoformat('2022-02-22')   ...:Out[2]:datetime.date(2022, 2, 22)

It supports IPython construct that plainPython does not understand (like magics):

In [3]:importtimeIn [4]:%pdoc time.sleepClass docstring:    sleep(seconds)    Delay execution for a given number of seconds.  The argument may be    a floating point number for subsecond precision.Call docstring:    Call self as a function.

This will also support top-level async when using IPython 7.0+

In [5]:importasyncio   ...:print('before')   ...:awaitasyncio.sleep(1)   ...:print('after')   ...:beforeafter

The namespace will persist across multiple code chucks, Let’s define a variable:

In [6]:who="World"

And now say hello:

In [7]:print('Hello,',who)Hello, World

If the current section raises an exception, you can add the:okexcept: flagto the current block, otherwise the build will fail.

In [8]:1/0---------------------------------------------------------------------------ZeroDivisionError                         Traceback (most recent call last)Cell In[8], line 1----> 1 1/0ZeroDivisionError: division by zero

IPython Sphinx directive module

To enable this directive, simply list it in your Sphinxconf.py file(making sure the directory where you placed it is visible to sphinx, as isneeded for all Sphinx directives). For example, to enable syntax highlightingand the IPython directive:

extensions=['IPython.sphinxext.ipython_console_highlighting','IPython.sphinxext.ipython_directive']

The IPython directive outputs code-blocks with the language ‘ipython’. Soif you do not have the syntax highlighting extension enabled as well, thenall rendered code-blocks will be uncolored. By default this directive assumesthat your prompts are unchanged IPython ones, but this can be customized.The configurable options that can be placed in conf.py are:

ipython_savefig_dir:

The directory in which to save the figures. This is relative to theSphinx source directory. The default ishtml_static_path.

ipython_rgxin:

The compiled regular expression to denote the start of IPython inputlines. The default isre.compile('In\[(\d+)\]:\s?(.*)\s*'). Youshouldn’t need to change this.

ipython_warning_is_error: [default to True]

Fail the build if something unexpected happen, for example if a block raisean exception but does not have the:okexcept: flag. The exact behavior ofwhat is considered strict, may change between the sphinx directive version.

ipython_rgxout:

The compiled regular expression to denote the start of IPython outputlines. The default isre.compile('Out\[(\d+)\]:\s?(.*)\s*'). Youshouldn’t need to change this.

ipython_promptin:

The string to represent the IPython input prompt in the generated ReST.The default is'In[%d]:'. This expects that the line numbers are usedin the prompt.

ipython_promptout:

The string to represent the IPython prompt in the generated ReST. Thedefault is'Out[%d]:'. This expects that the line numbers are usedin the prompt.

ipython_mplbackend:

The string which specifies if the embedded Sphinx shell should importMatplotlib and set the backend. The value specifies a backend that ispassed tomatplotlib.use() before any lines inipython_execlines areexecuted. If not specified in conf.py, then the default value of ‘agg’ isused. To use the IPython directive without matplotlib as a dependency, setthe value toNone. It may end up that matplotlib is still importedif the user specifies so inipython_execlines or makes use of the@savefig pseudo decorator.

ipython_execlines:

A list of strings to be exec’d in the embedded Sphinx shell. Typicalusage is to make certain packages always available. Set this to an emptylist if you wish to have no imports always available. If specified inconf.py asNone, then it has the effect of making no imports available.If omitted from conf.py altogether, then the default value of[‘import numpy as np’, ‘import matplotlib.pyplot as plt’] is used.

ipython_holdcount

When the @suppress pseudo-decorator is used, the execution count can beincremented or not. The default behavior is to hold the execution count,corresponding to a value ofTrue. Set this toFalse to incrementthe execution count after each suppressed command.

As an example, to use the IPython directive whenmatplotlib is not available,one sets the backend toNone:

ipython_mplbackend=None

An example usage of the directive is:

..ipython::    In [1]: x = 1    In [2]: y = x**2    In [3]: print(y)

Seehttp://matplotlib.org/sampledoc/ipython_directive.html for additionaldocumentation.

Pseudo-Decorators

Note: Only one decorator is supported per input. If more than one decoratoris specified, then only the last one is used.

In addition to the Pseudo-Decorators/options described at the above link,several enhancements have been made. The directive will emit a message to theconsole at build-time if code-execution resulted in an exception or warning.You can suppress these on a per-block basis by specifying the :okexcept:or :okwarning: options:

..ipython:::okexcept::okwarning:    In [1]: 1/0    In [2]: # raise warning.

To Do

  • Turn the ad-hoc test() function into a real test suite.

  • Break up ipython-specific functionality from matplotlib stuff into betterseparated code.