Source Files and Compilation

Cython source file names consist of the name of the module followed by a.pyx extension, for example a module called primes would have a sourcefile namedprimes.pyx.

Cython code, unlike Python, must be compiled. This happens in two stages:

  • A.pyx (or.py) file is compiled by Cython to a.c file.

  • The.c file is compiled by a C compiler to a.so file (or a.pyd file on Windows)

Once you have written your.pyx/.py file, there are a couple of wayshow to turn it into an extension module.

The following sub-sections describe several ways to build yourextension modules, and how to pass directives to the Cython compiler.

There are also a number of tools that process.pyx files apart from Cython, e.g.

Compiling from the command line

There are two ways of compiling from the command line.

  • Thecython command takes a.py or.pyx file andcompiles it into a C/C++ file.

  • Thecythonize command takes a.py or.pyx file andcompiles it into a C/C++ file. It then compiles the C/C++ file intoan extension module which is directly importable from Python.

Compiling with thecython command

One way is to compile it manually with the Cythoncompiler, e.g.:

$cythonprimes.pyx

This will produce a file calledprimes.c, which then needs to becompiled with the C compiler using whatever options are appropriate on yourplatform for generating an extension module. For these options look at theofficial Python documentation.

The other, and probably better, way is to use thesetuptools extensionprovided with Cython. The benefit of this method is that it will give theplatform specific compilation options, acting like a stripped down autotools.

Compiling with thecythonize command

Run thecythonize compiler command with your options and list of.pyx files to generate an extension module. For example:

$cythonize-a-iyourmod.pyx

This creates ayourmod.c file (oryourmod.cpp in C++ mode), compiles it,and puts the resulting extension module (.so or.pyd, depending on yourplatform) next to the source file for direct import (-i builds “in place”).The-a switch additionally produces an annotated html file of the source code.

Thecythonize command accepts multiple source files and glob patterns like**/*.pyx as argument and also understands the common-j option forrunning multiple parallel build jobs. When called without further options, itwill only translate the source files to.c or.cpp files. Pass the-h flag for a complete list of supported options.

There simpler command line toolcython only invokes the source code translator.

In the case of manual compilation, how to compile your.c files will varydepending on your operating system and compiler. The Python documentation forwriting extension modules should have some details for your system. On a Linuxsystem, for example, it might look similar to this:

$gcc-shared-pthread-fPIC-fwrapv-O2-Wall-fno-strict-aliasing\-I/usr/include/python3.5-oyourmod.soyourmod.c

(gcc will need to have paths to your included header files and pathsto libraries you want to link with.)

After compilation, ayourmod.so (yourmod.pyd for Windows)file is written into the target directoryand your module,yourmod, is available for you to import as with any otherPython module. Note that if you are not relying oncythonize orsetuptools,you will not automatically benefit from the platform specific file extensionthat CPython generates for disambiguation, such asyourmod.cpython-35m-x86_64-linux-gnu.so on a regular 64bit Linux installationof CPython 3.5.

Basicsetup.py

The setuptools extension provided with Cython allows you to pass.pyx filesdirectly to theExtension constructor in your setup file.

If you have a single Cython file that you want to turn into a compiledextension, say with filenameexample.pyx the associatedsetup.pywould be:

setup.py
fromsetuptoolsimportsetupfromCython.Buildimportcythonizesetup(ext_modules=cythonize("example.pyx"))

If your build depends directly on Cython in this way,then you may also want to inform pip thatCython is required forsetup.py to execute, followingPEP 518,creating apyproject.toml file containing, at least:

pyproject.toml
[build-system]requires=["setuptools","Cython"]

To understand thesetup.py more fully look at the officialsetuptools documentation.To compile the extension for use in the current directory use:

$pythonsetup.pybuild_ext--inplace

Note

setuptools 74.1.0 adds experimental support for extensions inpyproject.toml (instead ofsetup.py):

pyproject.toml
[build-system]requires=["setuptools","cython"]build-backend="setuptools.build_meta"[project]name="mylib-foo"version="0.42"[tool.setuptools]ext-modules=[{name="example",sources=["example.pyx"]}]

In this case, you can use any build frontend - e.g.build

$python-mbuild

Configuring the C-Build

Note

More details on building Cython modules that use cimport numpy can be found in theNumpy section of the user guide.

If you haveCython include files orCython definition files in non-standard places you can pass aninclude_path parameter tocythonize:

fromsetuptoolsimportsetupfromCython.Buildimportcythonizesetup(name="My hello app",ext_modules=cythonize("src/*.pyx",include_path=[...]),)

If you need to specify compiler options, libraries to link with or otherlinker options you will need to createExtension instances manually(note that glob syntax can still be used to specify multiple extensionsin one line):

fromsetuptoolsimportExtension,setupfromCython.Buildimportcythonizeextensions=[Extension("primes",["primes.pyx"],include_dirs=[...],libraries=[...],library_dirs=[...]),# Everything but primes.pyx is included here.Extension("*",["*.pyx"],include_dirs=[...],libraries=[...],library_dirs=[...]),]setup(name="My hello app",ext_modules=cythonize(extensions),)

Some useful options to know about are

  • include_dirs- list of directories to search for C/C++ header files (in Unix form for portability),

  • libraries - list of library names (not filenames or paths) to link against,

  • library_dirs - list of directories to search for C/C++ libraries at link time.

Note that when using setuptools, you should import it before Cython, otherwise,both might disagree about the class to use here.

Often, Python packages that offer a C-level API provide a way to findthe necessary C header files:

fromsetuptoolsimportExtension,setupfromCython.Buildimportcythonizeextensions=[Extension("*",["*.pyx"],include_dirs=["/usr/local/include"]),]setup(name="My hello app",ext_modules=cythonize(extensions),)

If your options are static (for example you do not need to call a tool likepkg-config to determine them) you can also provide them directly in your.pyx or.pxd source file using a special comment block at the start of the file:

# distutils: libraries = spam eggs# distutils: include_dirs = /opt/food/include

If you cimport multiple .pxd files defining libraries, then Cythonmerges the list of libraries, so this works as expected (similarlywith other options, likeinclude_dirs above).

If you have some C files that have been wrapped with Cython and you want tocompile them into your extension, you can define the setuptoolssourcesparameter:

# distutils: sources = [helper.c, another_helper.c]

Note that these sources are added to the list of sources of the currentextension module. Spelling this out in thesetup.py file looksas follows:

fromsetuptoolsimportExtension,setupfromCython.Buildimportcythonizesourcefiles=['example.pyx','helper.c','another_helper.c']extensions=[Extension("example",sourcefiles)]setup(ext_modules=cythonize(extensions))

TheExtension class takes many options, and a fuller explanation canbe found in thesetuptools documentation.

Sometimes this is not enough and you need finer customization of thesetuptoolsExtension.To do this, you can provide a custom functioncreate_extensionto create the finalExtension object after Cython has processedthe sources, dependencies and#distutils directives but before thefile is actually Cythonized.This function takes 2 argumentstemplate andkwds, wheretemplate is theExtension object given as input to Cythonandkwds is adict with all keywords which should be usedto create theExtension.The functioncreate_extension must return a 2-tuple(extension,metadata), whereextension is the createdExtension andmetadata is metadata which will be writtenas JSON at the top of the generated C files. This metadata is only usedfor debugging purposes, so you can put whatever you want in there(as long as it can be converted to JSON).The default function (defined inCython.Build.Dependencies) is:

defdefault_create_extension(template,kwds):if'depends'inkwds:include_dirs=kwds.get('include_dirs',[])+["."]depends=resolve_depends(kwds['depends'],include_dirs)kwds['depends']=sorted(set(depends+template.depends))t=template.__class__ext=t(**kwds)ifhasattr(template,"py_limited_api"):ext.py_limited_api=template.py_limited_apimetadata=dict(distutils=kwds,module_name=kwds['name'])returnext,metadata

In case that you pass a string instead of anExtension tocythonize(), thetemplate will be anExtension withoutsources. For example, if you docythonize("*.pyx"),thetemplate will beExtension(name="*.pyx",sources=[]).

Just as an example, this addsmylib as library to every extension:

fromCython.Build.Dependenciesimportdefault_create_extensiondefmy_create_extension(template,kwds):libs=kwds.get('libraries',[])+["mylib"]kwds['libraries']=libsreturndefault_create_extension(template,kwds)ext_modules=cythonize(...,create_extension=my_create_extension)

Note

If you Cythonize in parallel (using thenthreads argument),then the argument tocreate_extension must be pickleable.In particular, it cannot be a lambda function.

Cythonize arguments

The functioncythonize() can take extra arguments which will allow you tocustomize your build.

Cython.Build.cythonize(module_list,exclude=None,nthreads=0,aliases=None,quiet=False,force=None,language=None,exclude_failures=False,show_all_warnings=False,**options)

Compile a set of source modules into C/C++ files and return a list of distutilsExtension objects for them.

Parameters:

  • module_list – As module list, pass either a glob pattern, a list of globpatterns or a list of Extension objects. The latterallows you to configure the extensions separatelythrough the normal distutils options.You can also pass Extension objects that haveglob patterns as their sources. Then, cythonizewill resolve the pattern and create acopy of the Extension for every matching file.

  • exclude – When passing glob patterns asmodule_list, you can exclude certainmodule names explicitly by passing them into theexclude option.

  • nthreads – The number of concurrent builds for parallel compilation(requires themultiprocessing module).

  • aliases – If you want to use compiler directives like#distutils:... butcan only know at compile time (when running thesetup.py) which valuesto use, you can use aliases and pass a dictionary mapping those aliasesto Python strings when callingcythonize(). As an example, say youwant to use the compilerdirective#distutils:include_dirs=../static_libs/include/but this path isn’t always fixed and you want to find it when runningthesetup.py. You can then do#distutils:include_dirs=MY_HEADERS,find the value ofMY_HEADERS in thesetup.py, put it in a pythonvariable calledfoo as a string, and then callcythonize(...,aliases={'MY_HEADERS':foo}).

  • quiet – If True, Cython won’t print error, warning, or status messages during thecompilation.

  • force – Forces the recompilation of the Cython modules, even if the timestampsdon’t indicate that a recompilation is necessary.

  • language – To globally enable C++ mode, you can passlanguage='c++'. Otherwise, thiswill be determined at a per-file level based on compiler directives. Thisaffects only modules found based on file names. Extension instances passedintocythonize() will not be changed. It is recommended to ratheruse the compiler directive#distutils:language=c++ than this option.

  • exclude_failures – For a broad ‘try to compile’ mode that ignores compilationfailures and simply excludes the failed extensions,passexclude_failures=True. Note that this onlyreally makes sense for compiling.py files which can alsobe used without compilation.

  • show_all_warnings – By default, not all Cython warnings are printed.Set to true to show all warnings.

  • annotate – IfTrue, will produce a HTML file for each of the.pyx or.pyfiles compiled. The HTML file gives an indicationof how much Python interaction there is ineach of the source code lines, compared to plain C code.It also allows you to see the C/C++ codegenerated for each line of Cython code. This report is invaluable whenoptimizing a function for speed,and for determining when torelease the GIL:in general, anogil block may contain only “white” code.See examples inDetermining where to add types orPrimes.

  • annotate-fullc – IfTrue will produce a colorized HTML version ofthe source which includes entire generated C/C++-code.

  • compiler_directives – Allow to set compiler directives in thesetup.py like this:compiler_directives={'embedsignature':True}.SeeCompiler directives.

  • depfile – produce depfiles for the sources if True.

  • cache – IfTrue the cache enabled with default path. If the value is a path to a directory,then the directory is used to cache generated.c/.cpp files. By default cache is disabled.SeeCython cache.

Multiple Cython Files in a Package

To automatically compile multiple Cython files without listing all of themexplicitly, you can use glob patterns:

setup(ext_modules=cythonize("package/*.pyx"))

You can also use glob patterns inExtension objects if you passthem throughcythonize():

extensions=[Extension("*",["*.pyx"])]setup(ext_modules=cythonize(extensions))

Distributing Cython modules

Following recent improvements in the distribution toolchain, it isnot recommended to include generated files in source distributions.Instead,require Cython at build-time to generate the C/C++ files,as defined inPEP 518 andPEP 621. SeeBasic setup.py.

It is, however, possible to distribute the generated.c files together withyour Cython sources, so that users can install your module without needingto have Cython available.

Doing so allows you to make Cython compilation optional in theversion you distribute. Even if the user has Cython installed, they may notwant to use it just to install your module. Also, the installed versionmay not be the same one you used, and may not compile your sources correctly.

This simply means that thesetup.py file that you ship with will justbe a normal setuptools file on the generated.c files, for the basic examplewe would have instead:

fromsetuptoolsimportExtension,setupsetup(ext_modules=[Extension("example",["example.c"])])

This is easy to combine withcythonize() by changing the file extensionof the extension module sources:

fromsetuptoolsimportExtension,setupUSE_CYTHON=...# command line option, try-import, ...ext='.pyx'ifUSE_CYTHONelse'.c'extensions=[Extension("example",["example"+ext])]ifUSE_CYTHON:fromCython.Buildimportcythonizeextensions=cythonize(extensions)setup(ext_modules=extensions)

If you have many extensions and want to avoid the additional complexity in thedeclarations, you can declare them with their normal Cython sources and thencall the following function instead ofcythonize() to adapt the sourceslist in theExtensions when not using Cython:

importos.pathdefno_cythonize(extensions,**_ignore):forextensioninextensions:sources=[]forsfileinextension.sources:path,ext=os.path.splitext(sfile)ifextin('.pyx','.py'):ifextension.language=='c++':ext='.cpp'else:ext='.c'sfile=path+extsources.append(sfile)extension.sources[:]=sourcesreturnextensions

If you want to expose the C-level interface of your library for otherlibraries to cimport from, use package_data to install the.pxd files,e.g.:

setup(package_data={'my_package':['*.pxd'],'my_package/sub_package':['*.pxd'],},...)

These.pxd files need not have corresponding.pyxmodules if they contain purely declarations of external libraries.

Shared utility module

When a.pyx/.py file is compiled to.c file, Cython automatically embedsinternal utility code into the resulting.c file. For projects containing multipleCython modules, this can result in a larger total size of compiled extensions.To avoid redundant code, the common utility code can be extracted into a separate extensionmodule which is automatically cimported and used by the user-written extension modules.

Note

Currently, only memoryview utility code can be moved to the shared utility module.

Consider the following example package:

mypkg/+--__init__.py+--shared/|+--__init__.py|+--_cyutility.c+--subpkg1/+--__init__.py+--module.pyx

The_cyutility.c file contains the shared utility code andmodule.pyx isa standard Cython source file which will be compiled into an extension cimporting themypkg.shared._cyutility module (automatically).The compilation process now consist of three steps:

  1. Generating the shared utility code. This is done via the--generate-shared argument:

    $cython--generate-shared=mypkg/shared/_cyutility.c

  2. Translating all.pyx files to.c files with the--shared argument to providethe fully qualified name of the shared module:

    $cython--shared=mypkg.shared._cyutilitymodule.pyx

  3. Compiling the shared utility module and all other (user defined) extension modules.The shared utility module needs to be compiled as regular.c extension module,either by using the C compiler directly, or through setuptools, etc.

Warning

An extension module compiled with the--shared=... argument automaticallyimports the shared module under the fully qualified name provided via theargument parameter. Failing to import the shared module will cause a failureto import the extension module that uses it.

Compiling shared module using setuptools

Ifsetuptools is used in the build process, the fully qualified module nameof the shared utility module can be specified using theshared_utility_qualified_nameparameter ofcythonize() (instead of the--shared command line argument).To generate the extension sources of the shared module fromcythonize(),you need to explicitly pass anExtension object describing the module.Thesetup.py file would be:

setup.py
fromCython.BuildimportcythonizefromCython.CompilerimportOptionsfromsetuptoolsimportsetup,Extensionextensions=[Extension("*",["**/*.pyx"]),# Providing 'sources' is optional for the shared module.# If missing, the module package will be used for the path in 'build_dir'.Extension("mypkg.shared._cyutility",sources=["mypkg/shared/_cyutility.c"]),]setup(ext_modules=cythonize(extensions,shared_utility_qualified_name='mypkg.shared._cyutility'))

Integrating multiple modules

In some scenarios, it can be useful to link multiple Cython modules(or other extension modules) into a single binary, e.g. when embeddingPython in another application. This can be done through the inittabimport mechanism of CPython.

Create a new C file to integrate the extension modules and add thismacro to it:

#if PY_MAJOR_VERSION < 3# define MODINIT(name)  init ## name#else# define MODINIT(name)  PyInit_ ## name#endif

If you are only targeting Python 3.x, just usePyInit_ as prefix.

Then, for each of the modules, declare its module init functionas follows, replacingsome_module_name with the name of the module:

PyMODINIT_FUNCMODINIT(some_module_name)(void);

In C++, declare them asexternC.

If you are not sure of the name of the module init function, referto your generated module source file and look for a function namestarting withPyInit_.

Next, before you start the Python runtime from your application codewithPy_Initialize(), you need to initialise the modules at runtimeusing thePyImport_AppendInittab() C-API function, again insertingthe name of each of the modules:

PyImport_AppendInittab("some_module_name",MODINIT(some_module_name));

This enables normal imports for the embedded extension modules.

In order to prevent the joined binary from exporting all of the moduleinit functions as public symbols, Cython 0.28 and later can hide thesesymbols if the macroCYTHON_NO_PYINIT_EXPORT is defined whileC-compiling the module C files.

Also take a look at thecython_freeze tool.It can generate the necessary boilerplate code for linking one or moremodules into a single Python executable.

Compiling withpyximport

For building Cython modules during development without explicitlyrunningsetup.py after each change, you can usepyximport:

>>>importpyximport;pyximport.install()>>>importhelloworldHello World

This allows you to automatically run Cython on every.pyx thatPython is trying to import. You should use this for simple Cythonbuilds only where no extra C libraries and no special building setupis needed.

It is also possible to compile new.py modules that are beingimported (including the standard library and installed packages). Forusing this feature, just tell that topyximport:

>>>pyximport.install(pyimport=True)

In the case that Cython fails to compile a Python module,pyximportwill fall back to loading the source modules instead.

Note that it is not recommended to letpyximport build codeon end user side as it hooks into their import system. The best wayto cater for end users is to provide pre-built binary packages in thewheel packaging format.

Arguments

The functionpyximport.install() can take several arguments toinfluence the compilation of Cython or Python files.

pyximport.install(pyximport=True,pyimport=False,build_dir=None,build_in_temp=True,setup_args=None,reload_support=False,load_py_module_on_import_failure=False,inplace=False,language_level=None)

Main entry point for pyxinstall.

Call this to install the.pyx import hook inyour meta-path for a single Python process. If you want it to beinstalled whenever you use Python, add it to yoursitecustomize(as described above).

Parameters:

  • pyximport – If set to False, does not try to import.pyx files.

  • pyimport – You can passpyimport=True to alsoinstall the.py import hookin your meta-path. Note, however, that it is rather experimental,will not work at all for some.py files and packages, and willheavily slow down your imports due to search and compilation.Use at your own risk.

  • build_dir – By default, compiled modules will end up in a.pyxblddirectory in the user’s home directory. Passing a different pathasbuild_dir will override this.

  • build_in_temp – IfFalse, will produce the C files locally. Workingwith complex dependencies and debugging becomes more easy. Thiscan principally interfere with existing files of the same name.

  • setup_args – Dict of arguments for Distribution.Seedistutils.core.setup().

  • reload_support – Enables support for dynamicreload(my_module), e.g. after a change in the Cython code.Additional files<so_path>.reloadNN may arise on that account, whenthe previously loaded module file cannot be overwritten.

  • load_py_module_on_import_failure – If the compilation of a.pyfile succeeds, but the subsequent import fails for some reason,retry the import with the normal.py module instead of thecompiled module. Note that this may lead to unpredictable resultsfor modules that change the system state during their import, asthe second import will rerun these modifications in whatever statethe system was left after the import of the compiled modulefailed.

  • inplace – Install the compiled module(.so for Linux and Mac /.pyd for Windows)next to the source file.

  • language_level – The source language level to use: 2 or 3.The default is to use the language level of the current Pythonruntime for .py files and Py2 for.pyx files.

Dependency Handling

Sincepyximport does not usecythonize() internally, it currentlyrequires a different setup for dependencies. It is possible to declare thatyour module depends on multiple files, (likely.h and.pxd files).If your Cython module is namedfoo and thus has the filenamefoo.pyx then you should create another file in the same directorycalledfoo.pyxdep. Themodname.pyxdep file can be a list offilenames or “globs” (like*.pxd orinclude/*.h). Each filename orglob must be on a separate line. Pyximport will check the file date for eachof those files before deciding whether to rebuild the module. In order tokeep track of the fact that the dependency has been handled, Pyximport updatesthe modification time of your “.pyx” source file. Future versions may dosomething more sophisticated like informing setuptools of the dependenciesdirectly.

Limitations

pyximport does not usecythonize(). Thus it is notpossible to do things like using compiler directives atthe top of Cython files or compiling Cython code to C++.

Pyximport does not give you any control over how your Cython file iscompiled. Usually the defaults are fine. You might run into problems ifyou wanted to write your program in half-C, half-Cython and build theminto a single library.

Pyximport does not hide the setuptools/GCC warnings and errors generatedby the import process. Arguably this will give you better feedback ifsomething went wrong and why. And if nothing went wrong it will give youthe warm fuzzy feeling that pyximport really did rebuild your module as itwas supposed to.

Basic module reloading support is available with the optionreload_support=True.Note that this will generate a new module filename for each build and thusend up loading multiple shared libraries into memory over time. CPython has limitedsupport for reloading shared libraries as such,seePEP 489.

Pyximport puts both your.c file and the platform-specific binary intoa separate build directory, usually$HOME/.pyxblx/. To copy it backinto the package hierarchy (usually next to the source file) for manualreuse, you can pass the optioninplace=True.

Compiling withcython.inline

One can also compile Cython in a fashion similar to SciPy’sweave.inline.For example:

>>>importcython>>>deff(a):...ret=cython.inline("return a+b",b=3)...

Unbound variables are automatically pulled from the surrounding localand global scopes, and the result of the compilation is cached forefficient reuse.

Compiling withcython.compile

Cython supports transparent compiling of the cython code in a function using the@cython.compile decorator:

@cython.compiledefplus(a,b):returna+b

Parameters of the decorated function cannot have type declarations. Their types areautomatically determined from values passed to the function, thus leading to one or morespecialised compiled functions for the respective argument types.Executing example:

importcython@cython.compiledefplus(a,b):returna+bprint(plus('3','5'))print(plus(3,5))

will produce following output:

358

Compiling with Sage

The Sage notebook allows transparently editing and compiling Cythoncode simply by typing%cython at the top of a cell and evaluateit. Variables and functions defined in a Cython cell are imported into therunning session. Please checkSage documentation for details.

You can tailor the behavior of the Cython compiler by specifying thedirectives below.

Compiling with a Jupyter Notebook

It’s possible to compile code in a notebook cell with Cython.For this you need to load the Cython magic:

%load_extcython

Then you can define a Cython cell by writing%%cython on top of it.Like this:

%%cythoncdefinta=0foriinrange(10):a+=iprint(a)

Note that each cell will be compiled into a separate extension module. So if you use a package in a Cythoncell, you will have to import this package in the same cell. It’s not enough tohave imported the package in a previous cell. Cython will tell you that there are“undefined global names” at compilation time if you don’t comply.

The global names (top level functions, classes, variables and modules) of thecell are then loaded into the global namespace of the notebook. So in theend, it behaves as if you executed a Python cell.

Additional allowable arguments to the Cython magic are listed below.You can see them also by typing`%%cython? in IPython or a Jupyter notebook.

-a, –annotate

Produce a colorized HTML version of the source.

–annotate-fullc

Produce a colorized HTML version of the source which includes entire generated C/C++-code.

-+, –cplus

Output a C++ rather than C file.

-f, –force

Force the compilation of a new module, even if the source has been previously compiled.

-3

Select Python 3 syntax

-2

Select Python 2 syntax

-c=COMPILE_ARGS, –compile-args=COMPILE_ARGS

Extra flags to pass to compiler via the extra_compile_args.

–link-args LINK_ARGS

Extra flags to pass to linker via the extra_link_args.

-l LIB, –lib LIB

Add a library to link the extension against (can be specified multiple times).

-L dir

Add a path to the list of library directories (can be specified multiple times).

-I INCLUDE, –include INCLUDE

Add a path to the list of include directories (can be specified multiple times).

-S, –src

Add a path to the list of src files (can be specified multiple times).

-n NAME, –name NAME

Specify a name for the Cython module.

–pgo

Enable profile guided optimisation in the C compiler. Compiles the cell twice and executes it in between to generate a runtime profile.

–verbose

Print debug information like generated .c/.cpp file location and exact gcc/g++ command invoked.

Cython cache

The Cython cache is used to store cythonized.c/.cpp files to avoid running the Cython compiler on the files which were cythonized before.

Note

Only.c/.cpp files are cached. The C compiler is run every time. To avoid executing C compiler a tool like ccache needs to be used.

The Cython cache is disabled by default but can be enabled by thecache parameter ofcythonize():

fromsetuptoolsimportsetup,ExtensionfromCython.Buildimportcythonizeextensions=[Extension("*",["lib.pyx"]),]setup(name="hello",ext_modules=cythonize(extensions,cache=True))

The cached files are searched in the following paths by default in the following order:

  1. path specified in theCYTHON_CACHE_DIR environment variable,

  2. ~/Library/Caches/Cython on MacOS and$XDG_CACHE_HOME/cython on POSIX if theXDG_CACHE_HOME environment variable is defined,

  3. otherwise~/.cython.

Compiler options

Compiler options can be set in thesetup.py, before callingcythonize(),like this:

fromsetuptoolsimportsetupfromCython.BuildimportcythonizefromCython.CompilerimportOptionsOptions.docstrings=Falsesetup(name="hello",ext_modules=cythonize("lib.pyx"),)

Here are the options that are available:

Cython.Compiler.Options.docstrings=True

Whether or not to include docstring in the Python extension. If False, the binary sizewill be smaller, but the__doc__ attribute of any class or function will be anempty string.

Cython.Compiler.Options.embed_pos_in_docstring=False

Embed the source code position in the docstrings of functions and classes.

Cython.Compiler.Options.generate_cleanup_code=False

Decref global variables in each module on exit for garbage collection.0: None, 1+: interned objects, 2+: cdef globals, 3+: types objectsMostly for reducing noise in Valgrind as it typically executes at process exit(when all memory will be reclaimed anyways).Note that directly or indirectly executed cleanup code that makes use of globalvariables or types may no longer be safe when enabling the respective level sincethere is no guaranteed order in which the (reference counted) objects willbe cleaned up. The order can change due to live references and reference cycles.

Cython.Compiler.Options.clear_to_none=True

Should tp_clear() set object fields to None instead of clearing them to NULL?

Cython.Compiler.Options.annotate=False

Generate an annotated HTML version of the input source files for debugging and optimisation purposes.This has the same effect as theannotate argument incythonize().

Cython.Compiler.Options.fast_fail=False

This will abort the compilation on the first error occurred rather than tryingto keep going and printing further error messages.

Cython.Compiler.Options.warning_errors=False

Turn all warnings into errors.

Cython.Compiler.Options.error_on_unknown_names=True

Make unknown names an error. Python raises a NameError whenencountering unknown names at runtime, whereas this option makesthem a compile time error. If you want full Python compatibility,you should disable this option and also ‘cache_builtins’.

Cython.Compiler.Options.error_on_uninitialized=True

Make uninitialized local variable reference a compile time error.Python raises UnboundLocalError at runtime, whereas this option makesthem a compile time error. Note that this option affects only variablesof “python object” type.

Cython.Compiler.Options.convert_range=True

This will convert statements of the formforiinrange(...)toforifrom... wheni is a C integer type, and the direction(i.e. sign of step) can be determined.WARNING: This may change the semantics if the range causes assignment toi to overflow. Specifically, if this option is set, an error will beraised before the loop is entered, whereas without this option the loopwill execute until an overflowing value is encountered.

Cython.Compiler.Options.cache_builtins=True

Perform lookups on builtin names only once, at module initialisationtime. This will prevent the module from getting imported if abuiltin name that it uses cannot be found during initialisation.Default is True.Note that some legacy builtins are automatically remappedfrom their Python 2 names to their Python 3 names by Cythonwhen building in Python 3.x,so that they do not get in the way even if this option is enabled.

Cython.Compiler.Options.gcc_branch_hints=True

Generate branch prediction hints to speed up error handling etc.

Cython.Compiler.Options.lookup_module_cpdef=False

Enable this to allow one to writeyour_module.foo=... to overwrite thedefinition of the cpdef function foo, at the cost of an extra dictionarylookup on every call.If this is false it generates only the Python wrapper and no override check.

Cython.Compiler.Options.embed=None

Whether or not to embed the Python interpreter, for use in making astandalone executable or calling from external libraries.This will provide a C function which initialises the interpreter andexecutes the body of this module.Seethis demofor a concrete example.If true, the initialisation function is the C main() function, butthis option can also be set to a non-empty string to provide a function name explicitly.Default is False.

Cython.Compiler.Options.cimport_from_pyx=False

Allows cimporting from a pyx file without a pxd file.

Cython.Compiler.Options.buffer_max_dims=8

Maximum number of dimensions for buffers – set lower than number ofdimensions in numpy, asslices are passed by value and involve a lot of copying.

Cython.Compiler.Options.closure_freelist_size=8

Number of function closure instances to keep in a freelist (0: no freelists)

Compiler directives

Compiler directives are instructions which affect the behavior ofCython code. Here is the list of currently supported directives:

binding (True / False),default=True

Controls whether free functions behave more like Python’s CFunctions(e.g.len()) or, when set to True, more like Python’s functions.When enabled, functions will bind to an instance when looked up as aclass attribute (hence the name) and will emulate the attributesof Python functions, including introspections like argument names andannotations.

Changed in version 3.0.0:Default changed from False to True

boundscheck (True / False),default=True

If set to False, Cython is free to assume that indexing operations([]-operator) in the code will not cause any IndexErrors to beraised. Lists, tuples, and strings are affected only if the indexcan be determined to be non-negative (or ifwraparound is False).Conditions which would normally trigger an IndexError may instead causesegfaults or data corruption if this is set to False.

wraparound (True / False),default=True

In Python, arrays and sequences can be indexed relative to the end.For example, A[-1] indexes the last value of a list.In C, negative indexing is not supported.If set to False, Cython is allowed to neither check for nor correctlyhandle negative indices, possibly causing segfaults or data corruption.If bounds checks are enabled (the default, seeboundschecks above),negative indexing will usually raise anIndexError for indices thatCython evaluates itself.However, these cases can be difficult to recognise in user code todistinguish them from indexing or slicing that is evaluated by theunderlying Python array or sequence object and thus continues to supportwrap-around indices.It is therefore safest to apply this option only to code that does notprocess negative indices at all.

initializedcheck (True / False),default=True
If set to True, Cython checks that

  • a memoryview is initialized whenever its elements are accessedor assigned to.

  • a C++ class is initialized when it is accessed(only whencpp_locals is on)

Setting this to False disables these checks.

nonecheck (True / False),default=False

If set to False, Cython is free to assume that native fieldaccesses on variables typed as an extension type, or bufferaccesses on a buffer variable, never occurs when the variable isset toNone. Otherwise a check is inserted and theappropriate exception is raised. This is off by default forperformance reasons.

freethreading_compatible (True / False),default=False

If set to True, Cython sets thePy_mod_gil slot toPy_MOD_GIL_NOT_USED to signal that the module is safe to runwithout an active GIL and prevent the GIL from being enabledwhen the module is imported. Otherwise the slot is set toPy_MOD_GIL_USED which will cause the GIL to be automaticallyenabled. Setting this to True does not itself make the module safeto run without the GIL; it merely confirms that you have checkedthe logic and consider it safe to run. Since free-threading supportis still experimental itself, this is also an experimental directive thatmight be changed or removed in future releases.

subinterpreters_compatible (no / shared_gil / own_gil),default=no

If set toshared_gil orown_gil, then Cython sets thePy_mod_multiple_interpreters slot toPy_MOD_MULTIPLE_INTERPRETERS_SUPPORTEDorPy_MOD_PER_INTERPRETER_GIL_SUPPORTED respectively to signal thatthe module is safe to run in isolated subinterpreters. Setting this optiondoes not itself make the module safe to run in isolated subinterpreters;it merely confirms that you have checked the logic and consider it safe to run.Currentlycdef global variables (especially when the type is a Python object) andacquiring the GIL (but notre-acquiring the GIL) are known not to workcorrectly and will generate warnings at compile time.

overflowcheck (True / False),default=False

If set to True, raise errors on overflowing C integer arithmeticoperations. Incurs a modest runtime penalty, but is much faster thanusing Python ints.

overflowcheck.fold (True / False),default=True

If set to True, and overflowcheck is True, check the overflow bit fornested, side-effect-free arithmetic expressions once rather than at everystep. Depending on the compiler, architecture, and optimization settings,this may help or hurt performance. A simple suite of benchmarks can befound inDemos/overflow_perf.pyx.

embedsignature (True / False),default=False

If set to True, Cython will embed a textual copy of the callsignature in the docstring of all Python visible functions andclasses. Tools like IPython and epydoc can thus display thesignature, which cannot otherwise be retrieved aftercompilation.

embedsignature.format (c /python /clinic),default=”c”

If set toc, Cython will generate signatures preservingC type declarations and Python type annotations.If set topython, Cython will do a best attempt to usepure-Python type annotations in embedded signatures. For argumentswithout Python type annotations, the C type is mapped to theclosest Python type equivalent (e.g., Cshort is mapped toPythonint type and Cdouble is mapped to Pythonfloattype). The specific output and type mapping are experimental andmay change over time.Theclinic format generates signatures that are compatiblewith those understood by CPython’s Argument Clinic tool. TheCPython runtime strips these signatures from docstrings andtranslates them into a__text_signature__ attribute. This ismainly useful when usingbinding=False, since the Cythonfunctions generated withbinding=True do not have (nor need)a__text_signature__ attribute.

cdivision (True / False),default=False

If set to False, Cython will adjust the remainder and quotientoperators C types to match those of Python ints (which differ whenthe operands have opposite signs) and raise aZeroDivisionError when the right operand is 0. This has up toa 35% speed penalty. If set to True, no checks are performed. SeeCEP 516.

cdivision_warnings (True / False),default=False

If set to True, Cython will emit a runtime warning wheneverdivision is performed with negative operands. SeeCEP 516.

cpow (True / False),default=False

cpow modifies the return type ofa**b, as shown in thetable below:

cpow behaviour

Type ofa

Type ofb

cpow==True

cpow==False

C integer

Negative integer compile-time constant

Return type is C double

Return type is C double (special case)

C integer

C integer (known to be >= 0 at compile time)

Return type is integer

Return type is integer

C integer

C integer (may be negative)

Return type is integer

Return type is C double (note that Python would dynamically pickint orfloat here, while Cython doesn’t)

C floating point

C integer

Return type is floating point

Return type is floating point

C floating point (or C integer)

C floating point

Return type is floating point, result is NaN if the result would be complex

Either a C real or complex number at cost of some speed

Thecpow==True behaviour largely keeps the result type thesame as the operand types, while thecpow==False behaviourfollows Python and returns a flexible type depending on theinputs.

Introduced in Cython 3.0 with a default of False;before that, the behaviour matched thecpow=True version.

always_allow_keywords (True / False),default=True

When disabled, uses theMETH_NOARGS andMETH_O signatures whenconstructing functions/methods which take zero or one arguments. Has noeffect on special methods and functions with more than one argument. TheMETH_NOARGS andMETH_O signatures provide slightly fastercalling conventions but disallow the use of keywords.

c_api_binop_methods (True / False),default=False

When enabled, makes the special binary operator methods (__add__, etc.)behave according to the low-level C-API slot semantics, i.e. only a singlemethod implements both the normal and reversed operator. This used to bethe default in Cython 0.x and was now replaced by Python semantics, i.e. thedefault in Cython 3.x and later isFalse.

profile (True / False),default=False

Write hooks for Python profilers into the compiled C code.Whether the generated module actually uses profiling depends onthe value of the C macroCYTHON_PROFILE which is1 bydefault but which you can optionally set to0 to turn off theprofiling code at C compile-time. DefineCYTHON_USE_SYS_MONITORINGto either 1 or 0 to control the mechanism used to implement profilingon Python 3.13 and above. Note that neitherprofilenorlinetrace work with any tool that usessys.monitoringon Python 3.12.

linetrace (True / False),default=False

Write line tracing hooks for Python profilers or coverage reportinginto the compiled C code. This also enables profiling.Note that the generated module will not actually use linetracing, unless you additionally pass the C macro definitionCYTHON_TRACE=1 to the C compiler (e.g. using the setuptools optiondefine_macros). DefineCYTHON_TRACE_NOGIL=1 to also includenogil functions and sections. DefineCYTHON_USE_SYS_MONITORINGto either 1 or 0 to control the mechanism used to implement thesefeatures on Python 3.13 and above. Note that neitherprofilenorlinetrace work with any tool that usessys.monitoringon Python 3.12.

infer_types (True / False),default=None

Infer types of untyped variables in function bodies. Default isNone, indicating that only safe (semantically-unchanging) inferencesare allowed.In particular, inferringintegral types for variablesused in arithmeticexpressions is considered unsafe (due to possible overflow) and must beexplicitly requested.

language_level (2/3/3str),default=None

Globally set the Python language level to be used for module compilation.Default is None, indicating compatibility with Python 3 in Cython 3.x and with Python 2in Cython 0.x. To enable Python 3 source code semantics, set this to 3 (or 3str) at the startof a module or pass the “-3” or “–3str” command line options to thecompiler. For Python 2 semantics, use 2 and “-2” accordingly.Before Cython 3.1, the3str option enabled Python 3 semantics but didnot change thestr type and unprefixed string literals tounicode when the compiled code runs in Python 2.x.In Cython 3.1,3str is an alias for3.Language level 2 ignoresx:int type annotations due to the int/long ambiguity.Note that cimported files inherit this setting from the modulebeing compiled, unless they explicitly set their own language level.Included source files always inherit this setting.

c_string_type (bytes / str / unicode),default=bytes

Globally set the type of an implicit coercion from char* or std::string.

c_string_encoding (ascii, default, utf-8, etc.),default=””

Globally set the encoding to use when implicitly coercing char* or std:stringto a unicode object. Coercion from a unicode object to C type is only allowedwhen set toascii ordefault, the latter being utf-8 in Python 3 andnearly-always ascii in Python 2.

type_version_tag (True / False),default=True

Enables the attribute cache for extension types in CPython by setting thetype flagPy_TPFLAGS_HAVE_VERSION_TAG. Default is True, meaning thatthe cache is enabled for Cython implemented types. To disable itexplicitly in the rare cases where a type needs to juggle with itstp_dictinternally without paying attention to cache consistency, this option canbe set to False. Note that this no longer applies to Python 3.11 and later.

unraisable_tracebacks (True / False),default=False

Whether to print tracebacks when suppressing unraisable exceptions.

iterable_coroutine (True / False),default=False

PEP 492 specifies that async-defcoroutines must not be iterable, in order to prevent accidental misuse innon-async contexts. However, this makes it difficult and inefficient to writebackwards compatible code that uses async-def coroutines in Cython but needs tointeract with async Python code that uses the older yield-from syntax, such asasyncio before Python 3.5. This directive can be applied in modules orselectively as decorator on an async-def coroutine to make the affectedcoroutine(s) iterable and thus directly interoperable with yield-from.

annotation_typing (True / False),default=True

Uses function argument annotations to determine the type of variables.Since Python does not enforce types given inannotations, setting to False gives greater compatibility with Python code.From Cython 3.0,annotation_typing can be set on a per-function orper-class basis.

emit_code_comments (True / False),default=True

Copy the original source code line by line into C code comments in the generatedcode file to help with understanding the output.This is also required for coverage analysis.

cpp_locals (True / False),default=False

Make C++ variables behave more like Python variables by allowing them to be“unbound” instead of always default-constructing them at the start of afunction. Seecpp_locals directive for more detail.

legacy_implicit_noexcept (True / False),default=False

When enabled,cdef functions will not propagate raised exceptions by default. Hence,the function will behave in the same way as if declared withnoexcept keyword. SeeError return values for details. Setting this directive toTrue willcause Cython 3.0 to have the same semantics as Cython 0.x. This directive was solely addedto help migrate legacy code written before Cython 3. It will be removed in a future release.

Configurable optimisations

optimize.use_switch (True / False),default=True

Whether to expand chained if-else statements (including statements likeifx==1orx==2:) into C switch statements. This can have performancebenefits if there are lots of values but cause compiler errors if there are anyduplicate values (which may not be detectable at Cython compile time for allC constants).

optimize.unpack_method_calls (True / False),default=True

Cython can generate code that optimistically checks for Python method objectsat call time and unpacks the underlying function to call it directly. Thiscan substantially speed up method calls, especially for builtins, but may alsohave a slight negative performance impact in some cases where the guess goescompletely wrong.Disabling this option can also reduce the code size.

Warnings

All warning directives take True / False as optionsto turn the warning on / off.

warn.undeclared (default False)

Warns about any variables that are implicitly declared without acdef declaration

warn.unreachable (default True)

Warns about code paths that are statically determined to be unreachable, e.g.returning twice unconditionally.

warn.maybe_uninitialized (default False)

Warns about use of variables that are conditionally uninitialized.

warn.unused (default False)

Warns about unused variables and declarations

warn.unused_arg (default False)

Warns about unused function arguments

warn.unused_result (default False)

Warns about unused assignment to the same name, such asr=2;r=1+2

warn.multiple_declarators (default True)

Warns about multiple variables declared on the same line with at least one pointer type.For examplecdefdouble*a,b - which, as in C, declaresa as a pointer,b asa value type, but could be mininterpreted as declaring two pointers.

warn.deprecated.DEF (default False)

Warns about use of the deprecatedDEF statement in Cython code, seeConditional Compilation andDeprecation of DEF / IF.

warn.deprecated.IF (default True)

Warns about use of the deprecatedIF statement in Cython code, seeConditional Compilation andDeprecation of DEF / IF.

show_performance_hints (default True)

Show performance hints during compilation pointing to places in the code which can yield performance degradation.Note that performance hints are not warnings and hence the directives starting withwarn. above do not affect themand they will not trigger a failure when “error on warnings” is enabled.

How to set directives

Globally

One can set compiler directives through a special header comment near the top of the file, like this:

# cython: language_level=3, boundscheck=False

The comment must appear before any code (but can appear after othercomments or whitespace).

One can also pass a directive on the command line by using the -X switch:

$cython-Xboundscheck=True...

Directives passed on the command line will override directives set inheader comments.

Locally

For local blocks, you need to cimport the special builtincythonmodule:

#!pythoncimportcython

Then you can use the directives either as decorators or in a withstatement, like this:

#!python@cython.boundscheck(False)# turn off boundscheck for this functiondeff():...# turn it temporarily on again for this blockwithcython.boundscheck(True):...

Warning

These two methods of setting directives arenotaffected by overriding the directive on the command-line using the-X option.

Insetup.py

Compiler directives can also be set in thesetup.py file by passing a keywordargument tocythonize:

fromsetuptoolsimportsetupfromCython.Buildimportcythonizesetup(name="My hello app",ext_modules=cythonize('hello.pyx',compiler_directives={'embedsignature':True}),)

This will override the default directives as specified in thecompiler_directives dictionary.Note that explicit per-file or local directives as explained above take precedence over thevalues passed tocythonize.

C line numbers in tracebacks

To provide more detailed debug information, Python tracebacks of Cython modulesshow the C line where the exception originated (or was propagated). This feature is notentirely for free and can visibly increase the C compile time as well as adding 0-5% to thesize of the binary extension module. It is therefore disabled in Cython 3.1 and can be controlled using C macros.

  • CYTHON_CLINE_IN_TRACEBACK=1 always shows the C line number in tracebacks,

  • CYTHON_CLINE_IN_TRACEBACK=0 never shows the C line number in tracebacks,

Unless the feature is disabled completely with this macro, there is also support for enabling and disablingthe feature at runtime, at the before mentioned cost of longer C compile times and larger extension modules.This can be configured with the C macro

CYTHON_CLINE_IN_TRACEBACK_RUNTIME=1

To then change the behaviour at runtime, you can import the special modulecython_runtimeafter loading a Cython module and set the attributecline_in_traceback in that moduleto either true or false to control the behaviour as your Cython code is being run:

importcython_runtimecython_runtime.cline_in_traceback=TrueraiseValueError(5)

If both macros arenot defined by the build setup orCFLAGS, the feature is disabled.

In Cython 3.0 and earlier, the Cython compiler optionc_line_in_traceback (passed asan argument tocythonize insetup.py) or the commandline argument--no-c-in-traceback could also be used to disable this feature.From Cython 3.1, this is still possible, but should be migrated to using the C macros instead.Before Cython 3.1, theCYTHON_CLINE_IN_TRACEBACK macro already works as describedbut the Cython option is needed to remove the compile-time cost.

C macro defines

Cython has a number of C macros that can be used to control compilation. Typically, thesewould be set usingextra_compile_args insetup.py (for exampleextra_compile_args=['-DCYTHON_USE_TYPE_SPECS=1']), however they can also be set inother ways like using theCFLAGS environmental variable.

These macros are set automatically by Cython to sensible default values unlessyou chose to explicitly override them, so they are a tool that most userscan happily ignore. Not all combinations of macros are compatible or tested, andsome change the default value of other macros. They are listed below in rough order frommost important to least important:

Py_LIMITED_API

Turns on Cython’s Limited API support, meaning that one compiled modulecan be used by many Python interpreter versions (at the cost of some performance).At this stage many features do not work in the Limited API. You should set thismacro to be the version hex for theminimum Python version you want to support (>=3.8).0x03080000 will supportPython 3.8 upwards.Note that this is aPythonmacro,rather than just a Cython macro, and so it changes what parts of the Python headersare visible too. SeeThe Limited API and Stable ABI for more details about this feature.

CYTHON_PEP489_MULTI_PHASE_INIT

Uses multi-phase module initialization as described inPEP 489. This improvesPython compatibility, especially when running the initial import of the code when itmakes attributes such as__file__ available. It is therefore on by defaultwhere supported.

CYTHON_USE_MODULE_STATE

Stores module data on a struct associated with the module object rather than asC global variables. The advantage is that it should be possible to import thesame module more than once (e.g. in different sub-interpreters). At the momentthis is experimental and not all data has been moved. Specifically,cdefglobals have not been moved.

CYTHON_USE_TYPE_SPECS

Definescdefclasses asHeap Types rather than“static types”. Practically this does not change a lot from a userpoint of view, but it is needed to implement Limited API support.

CYTHON_PROFILE,CYTHON_TRACE,CYTHON_TRACE_NOGIL

These control the inclusion of profiling and line tracing calls in the module.See theprofile andlinetraceCompiler directives.CYTHON_PROFILE is on by default; theCYTHON_TRACE macros areoff by default.

CYTHON_USE_SYS_MONITORING

On Python 3.13+ this selects the newsys.monitoringmechanism for profiling and linetracing. It is on by default, but can be set to 0to force use of the old mechanism. Some tools still require the old mechanism,most notably “Coverage” (as of 2025).

CYTHON_EXTERN_C

Slightly different to the other macros, this controls howcdefpublicfunctions appear to C++ code. SeeC++ public declarations for full details.

CYTHON_CLINE_IN_TRACEBACK

Controls whether C lines numbers appear in tracebacks.SeeC line numbers in tracebacks for a complete description.

CYTHON_CCOMPLEX

Passes complex numbers using the C or C++ language standard library typesinstead of an internal type defined by Cython. Turning it onmaximizes compatibility with external libraries. However, MSVChas poor standards support (especially in C mode) and so struggles touse the standard library types. It is on by default on platformswhere we think it’s likely to work.

CYTHON_COMPRESS_STRINGS

Store Python strings in the binary module as compressed data, decompressing themat import time. By default,zlib compression is used (CYTHON_COMPRESS_STRINGS=1).Set to0 to disable compression or to2 to selectbzip2 compression.Note that the respective standard library decompression module must be availableat module import time, or the import will fail.compression.zstd can be selected withCYTHON_COMPRESS_STRINGS=3 but is onlyavailable in the standard library in Python 3.14 and later. Cython will thenfall back tozlib when compiling in older Python versions.

CYTHON_IMMORTAL_CONSTANTS

Makes cached constants (e.g. strings, tuples, ints, floats, slices) immortal,in Python versions that support immortality. This is most useful whenthe constants are used in many different threads because it avoids most writesto the constants due to reference counting. Disabled by default, but enabledin free-threaded builds.

There is a further list of macros which turn off various optimizations or languagefeatures. Under normal circumstance Cython enables these automatically based on theversion of Python you are compiling for so there is no need to use themto try to enable extra optimizations - all supported optimizations are enabled bydefault. These are mostly relevant if you’re tying to get Cython working in anew and unsupported Python interpreter where you will typically want to setthem to 0 todisable optimizations. They are listed below for completeness buthidden by default since most users will be uninterested in changing them.