Using Sphinx for kernel documentation

The Linux kernel usesSphinx to generate pretty documentation fromreStructuredText files underDocumentation. To build the documentation inHTML or PDF formats, usemakehtmldocs ormakepdfdocs. The generateddocumentation is placed inDocumentation/output.

The reStructuredText files may contain directives to include structureddocumentation comments, or kernel-doc comments, from source files. Usually theseare used to describe the functions and types and design of the code. Thekernel-doc comments have some special structure and formatting, but beyond thatthey are also treated as reStructuredText.

Finally, there are thousands of plain text documentation files scattered aroundDocumentation. Some of these will likely be converted to reStructuredTextover time, but the bulk of them will remain in plain text.

Sphinx Install

The ReST markups currently used by the Documentation/ files are meant to bebuilt withSphinx version 3.4.3 or higher.

There’s a script that checks for the Sphinx requirements. Please seeChecking for Sphinx dependencies for further details.

Most distributions are shipped with Sphinx, but its toolchain is fragile,and it is not uncommon that upgrading it or some other Python packageson your machine would cause the documentation build to break.

A way to avoid that is to use a different version than the one shippedwith your distributions. In order to do so, it is recommended to installSphinx inside a virtual environment, usingvirtualenv-3orvirtualenv, depending on how your distribution packaged Python 3.

In summary, if you want to install the latest version of Sphinx, youshould do:

$ virtualenv sphinx_latest$ . sphinx_latest/bin/activate(sphinx_latest) $ pip install -r Documentation/sphinx/requirements.txt

After running.sphinx_latest/bin/activate, the prompt will change,in order to indicate that you’re using the new environment. If youopen a new shell, you need to rerun this command to enter again atthe virtual environment before building the documentation.

Image output

The kernel documentation build system contains an extension thathandles images in both GraphViz and SVG formats (seeFigures & Images).

For it to work, you need to install both GraphViz and ImageMagickpackages. If those packages are not installed, the build system willstill build the documentation, but won’t include any images at theoutput.

PDF and LaTeX builds

Such builds are currently supported only with Sphinx versions 2.4 and higher.

For PDF and LaTeX output, you’ll also needXeLaTeX version 3.14159265.

Depending on the distribution, you may also need to install a series oftexlive packages that provide the minimal set of functionalitiesrequired forXeLaTeX to work.

Math Expressions in HTML

Some ReST pages contain math expressions. Due to the way Sphinx works,those expressions are written using LaTeX notation.There are two options for Sphinx to render math expressions in html output.One is an extension calledimgmath which converts math expressions intoimages and embeds them in html pages.The other is an extension calledmathjax which delegates math renderingto JavaScript capable web browsers.The former was the only option for pre-6.1 kernel documentation and itrequires quite a few texlive packages including amsfonts and amsmath amongothers.

Since kernel release 6.1, html pages with math expressions can be builtwithout installing any texlive packages. SeeChoice of Math Renderer forfurther info.

Checking for Sphinx dependencies

There’s a script that automatically checks for Sphinx dependencies. If it canrecognize your distribution, it will also give a hint about the installcommand line options for your distro:

$ ./tools/docs/sphinx-pre-installChecking if the needed tools for Fedora release 26 (Twenty Six) are availableWarning: better to also install "texlive-luatex85".You should run:        sudo dnf install -y texlive-luatex85        /usr/bin/virtualenv sphinx_2.4.4        . sphinx_2.4.4/bin/activate        pip install -r Documentation/sphinx/requirements.txtCan't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.

By default, it checks all the requirements for both html and PDF, includingthe requirements for images, math expressions and LaTeX build, and assumesthat a virtual Python environment will be used. The ones needed for htmlbuilds are assumed to be mandatory; the others to be optional.

It supports two optional parameters:

--no-pdf

Disable checks for PDF;

--no-virtualenv

Use OS packaging for Sphinx instead of Python virtual environment.

Installing Sphinx Minimal Version

When changing Sphinx build system, it is important to ensure thatthe minimal version will still be supported. Nowadays, it isbecoming harder to do that on modern distributions, as it is notpossible to install with Python 3.13 and above.

Testing with the lowest supported Python version as defined atMinimal requirements to compile the Kernel can be done by creatinga venv with it with, and install minimal requirements with:

/usr/bin/python3.9 -m venv sphinx_min. sphinx_min/bin/activatepip install -r Documentation/sphinx/min_requirements.txt

A more comprehensive test can be done by using:

tools/docs/test_doc_build.py

Such script create one Python venv per supported version,optionally building documentation for a range of Sphinx versions.

Sphinx Build

The usual way to generate the documentation is to runmakehtmldocs ormakepdfdocs. There are also other formats available: see the documentationsection ofmakehelp. The generated documentation is placed informat-specific subdirectories underDocumentation/output.

To generate documentation, Sphinx (sphinx-build) must obviously beinstalled. For PDF output you’ll also needXeLaTeX andconvert(1)from ImageMagick (https://www.imagemagick.org).[1] All of these arewidely available and packaged in distributions.

To pass extra options to Sphinx, you can use theSPHINXOPTS makevariable. For example, usemakeSPHINXOPTS=-vhtmldocs to get more verboseoutput.

It is also possible to pass an extra DOCS_CSS overlay file, in order to customizethe html layout, by using theDOCS_CSS make variable.

By default, the “Alabaster” theme is used to build the HTML documentation;this theme is bundled with Sphinx and need not be installed separately.The Sphinx theme can be overridden by using theDOCS_THEME make variable.

Note

Some people might prefer to use the RTD theme for html output.Depending on the Sphinx version, it should be installed separately,withpipinstallsphinx_rtd_theme.

There is another make variableSPHINXDIRS, which is useful when testbuilding a subset of documentation. For example, you can build documentsunderDocumentation/doc-guide by runningmakeSPHINXDIRS=doc-guidehtmldocs.The documentation section ofmakehelp will show you the list ofsubdirectories you can specify.

To remove the generated documentation, runmakecleandocs.

[1]

Havinginkscape(1) from Inkscape (https://inkscape.org)as well would improve the quality of images embedded in PDFdocuments, especially for kernel releases 5.18 and later.

Choice of Math Renderer

Since kernel release 6.1, mathjax works as a fallback math renderer forhtml output.[2]

Math renderer is chosen depending on available commands as shown below:

Math Renderer Choices for HTML

Math renderer

Required commands

Image format

imgmath

latex, dvipng

PNG (raster)

mathjax

The choice can be overridden by setting an environment variableSPHINX_IMGMATH as shown below:

Effect of SettingSPHINX_IMGMATH

Setting

Renderer

SPHINX_IMGMATH=yes

imgmath

SPHINX_IMGMATH=no

mathjax

[2]

Fallback of math renderer requires Sphinx >=1.8.

Writing Documentation

Adding new documentation can be as simple as:

  1. Add a new.rst file somewhere underDocumentation.

  2. Refer to it from the Sphinx mainTOC tree inDocumentation/index.rst.

This is usually good enough for simple documentation (like the one you’rereading right now), but for larger documents it may be advisable to create asubdirectory (or use an existing one). For example, the graphics subsystemdocumentation is underDocumentation/gpu, split to several.rst files,and has a separateindex.rst (with atoctree of its own) referenced fromthe main index.

See the documentation forSphinx andreStructuredText on what you can dowith them. In particular, the SphinxreStructuredText Primer is a good placeto get started with reStructuredText. There are also someSphinx specificmarkup constructs.

Specific guidelines for the kernel documentation

Here are some specific guidelines for the kernel documentation:

  • Please don’t go overboard with reStructuredText markup. Keep itsimple. For the most part the documentation should be plain text withjust enough consistency in formatting that it can be converted toother formats.

  • Please keep the formatting changes minimal when converting existingdocumentation to reStructuredText.

  • Also update the content, not just the formatting, when convertingdocumentation.

  • Please stick to this order of heading adornments:

    1. = with overline for document title:

      ==============Document title==============
    2. = for chapters:

      Chapters========
    3. - for sections:

      Section-------
    4. ~ for subsections:

      Subsection~~~~~~~~~~

    Although RST doesn’t mandate a specific order (“Rather than imposing a fixednumber and order of section title adornment styles, the order enforced will bethe order as encountered.”), having the higher levels the same overall makesit easier to follow the documents.

  • For inserting fixed width text blocks (for code examples, use caseexamples, etc.), use:: for anything that doesn’t really benefitfrom syntax highlighting, especially short snippets. Use..code-block::<language> for longer code blocks that benefitfrom highlighting. For a short snippet of code embedded in the text, use ``.

The C domain

TheSphinx C Domain (name c) is suited for documentation of C API. E.g. afunction prototype:

..c:function:: int ioctl( int fd, int request )

The C domain of the kernel-doc has some additional features. E.g. you canrename the reference name of a function with a common name likeopen orioctl:

..c:function:: int ioctl( int fd, int request ):name: VIDIOC_LOG_STATUS

The func-name (e.g. ioctl) remains in the output but the ref-name changed fromioctl toVIDIOC_LOG_STATUS. The index entry for this function is alsochanged toVIDIOC_LOG_STATUS.

Please note that there is no need to usec:func: to generate crossreferences to function documentation. Due to some Sphinx extension magic,the documentation build system will automatically turn a reference tofunction() into a cross reference if an index entry for the givenfunction name exists. If you seec:func: use in a kernel document,please feel free to remove it.

Tables

ReStructuredText provides several options for table syntax. Kernel style fortables is to prefersimple table syntax orgrid table syntax. See thereStructuredText user reference for table syntax for more details.

list tables

The list-table formats can be useful for tables that are not easily laidout in the usual Sphinx ASCII-art formats. These formats are nearlyimpossible for readers of the plain-text documents to understand, though,and should be avoided in the absence of a strong justification for theiruse.

Theflat-table is a double-stage list similar to thelist-table withsome additional features:

  • column-span: with the rolecspan a cell can be extended throughadditional columns

  • row-span: with the rolerspan a cell can be extended throughadditional rows

  • auto span rightmost cell of a table row over the missing cells on the rightside of that table-row. With Option:fill-cells: this behavior canchanged fromauto span toauto fill, which automatically inserts (empty)cells instead of spanning the last cell.

options:

  • :header-rows: [int] count of header rows

  • :stub-columns: [int] count of stub columns

  • :widths: [[int] [int] ... ] widths of columns

  • :fill-cells: instead of auto-spanning missing cells, insert missing cells

roles:

  • :cspan: [int] additional columns (morecols)

  • :rspan: [int] additional rows (morerows)

The example below shows how to use this markup. The first level of the stagedlist is thetable-row. In thetable-row there is only one markup allowed,the list of the cells in thistable-row. Exceptions arecomments (.. )andtargets (e.g. a ref to:ref:`lastrow<lastrow>` /last row).

..flat-table:: table title:widths: 2 1 1 3* - head col 1- head col 2- head col 3- head col 4* - row 1- field 1.1- field 1.2 with autospan* - row 2- field 2.1-:rspan:`1`:cspan:`1` field 2.2 - 3.3* .. _`last row`:- row 3

Rendered as:

table title

head col 1

head col 2

head col 3

head col 4

row 1

field 1.1

field 1.2 with autospan

row 2

field 2.1

field 2.2 - 3.3

row 3

Cross-referencing

Cross-referencing from one documentation page to another can be done simply bywriting the path to the document file, no special syntax required. The path canbe either absolute or relative. For absolute paths, start it with“Documentation/”. For example, to cross-reference to this page, all thefollowing are valid options, depending on the current document’s directory (notethat the.rst extension is required):

See Documentation/doc-guide/sphinx.rst. This always works.Take a look at sphinx.rst, which is at this same directory.Read ../sphinx.rst, which is one directory above.

If you want the link to have a different rendered text other than the document’stitle, you need to use Sphinx’sdoc role. For example:

See :doc:`my custom link text for document sphinx <sphinx>`.

For most use cases, the former is preferred, as it is cleaner and more suitedfor people reading the source files. If you come across a:doc: usage thatisn’t adding any value, please feel free to convert it to just the documentpath.

For information on cross-referencing to kernel-doc functions or types, seeWriting kernel-doc comments.

Referencing commits

References to git commits are automatically hyperlinked given that they arewritten in one of these formats:

commit 72bf4f1767f0commit 72bf4f1767f0 ("net: do not leave an empty skb in write queue")

Figures & Images

If you want to add an image, you should use thekernel-figure andkernel-image directives. E.g. to insert a figure with a scalableimage format, use SVG (SVG image example):

.. kernel-figure::  svg_image.svg   :alt:    simple SVG image   SVG image example
simple SVG image

SVG image example

The kernel figure (and image) directive supportsDOT formatted files, see

A simple example (DOT’s hello world example):

.. kernel-figure::  hello.dot   :alt:    hello world   DOT's hello world example
graphG{Hello--World}

DOT’s hello world example

Embeddedrender markups (or languages) like Graphviz’sDOT are provided by thekernel-render directives.:

.. kernel-render:: DOT   :alt: foobar digraph   :caption: Embedded **DOT** (Graphviz) code   digraph foo {    "bar" -> "baz";   }

How this will be rendered depends on the installed tools. If Graphviz isinstalled, you will see a vector image. If not, the raw markup is inserted asliteral-block (Embedded DOT (Graphviz) code).

digraph foo {   "bar" -> "baz";}

EmbeddedDOT (Graphviz) code

Therender directive has all the options known from thefigure directive,plus optioncaption. Ifcaption has a value, afigure node isinserted. If not, animage node is inserted. Acaption is also needed, ifyou want to refer to it (Embedded SVG markup).

EmbeddedSVG:

.. kernel-render:: SVG   :caption: Embedded **SVG** markup   :alt: so-nw-arrow   <?xml version="1.0" encoding="UTF-8"?>   <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>      ...   </svg>
so-nw-arrow

EmbeddedSVG markup