2.1.Listing whole packages¶

Thepackages option tells the Distutils to process (build, distribute,install, etc.) all pure Python modules found in each package mentioned in thepackages list. In order to do this, of course, there has to be acorrespondence between package names and directories in the filesystem. Thedefault correspondence is the most obvious one, i.e. packagedistutils isfound in the directorydistutils relative to the distribution root.Thus, when you saypackages=['foo'] in your setup script, you arepromising that the Distutils will find a filefoo/__init__.py (whichmight be spelled differently on your system, but you get the idea) relative tothe directory where your setup script lives. If you break this promise, theDistutils will issue a warning but still process the broken package anyway.

If you use a different convention to lay out your source directory, that’s noproblem: you just have to supply thepackage_dir option to tell theDistutils about your convention. For example, say you keep all Python sourceunderlib, so that modules in the “root package” (i.e., not in anypackage at all) are inlib, modules in thefoo package are inlib/foo, and so forth. Then you would put

package_dir={'':'lib'}

in your setup script. The keys to this dictionary are package names, and anempty package name stands for the root package. The values are directory namesrelative to your distribution root. In this case, when you saypackages=['foo'], you are promising that the filelib/foo/__init__.py exists.

Another possible convention is to put thefoo package right inlib, thefoo.bar package inlib/bar, etc. This would bewritten in the setup script as

package_dir={'foo':'lib'}

Apackage:dir entry in thepackage_dir dictionary implicitlyapplies to all packages belowpackage, so thefoo.bar case isautomatically handled here. In this example, havingpackages=['foo','foo.bar'] tells the Distutils to look forlib/__init__.py andlib/bar/__init__.py. (Keep in mind that althoughpackage_dirapplies recursively, you must explicitly list all packages inpackages: the Distutils willnot recursively scan your source treelooking for any directory with an__init__.py file.)

2.2.Listing individual modules¶

For a small module distribution, you might prefer to list all modules ratherthan listing packages—especially the case of a single module that goes in the“root package” (i.e., no package at all). This simplest case was shown insectionA Simple Example; here is a slightly more involved example:

py_modules=['mod1','pkg.mod2']

This describes two modules, one of them in the “root” package, the other in thepkg package. Again, the default package/directory layout implies thatthese two modules can be found inmod1.py andpkg/mod2.py, andthatpkg/__init__.py exists as well. And again, you can override thepackage/directory correspondence using thepackage_dir option.

2.3.Describing extension modules¶

Just as writing Python extension modules is a bit more complicated than writingpure Python modules, describing them to the Distutils is a bit more complicated.Unlike pure modules, it’s not enough just to list modules or packages and expectthe Distutils to go out and find the right files; you have to specify theextension name, source file(s), and any compile/link requirements (includedirectories, libraries to link with, etc.).

All of this is done through another keyword argument tosetup(), theext_modules option.ext_modules is just a list ofExtension instances, each of which describes asingle extension module.Suppose your distribution includes a single extension, calledfoo andimplemented byfoo.c. If no additional instructions to thecompiler/linker are needed, describing this extension is quite simple:

Extension('foo',['foo.c'])

TheExtension class can be imported fromdistutils.core alongwithsetup(). Thus, the setup script for a module distribution thatcontains only this one extension and nothing else might be:

fromdistutils.coreimportsetup,Extensionsetup(name='foo',version='1.0',ext_modules=[Extension('foo',['foo.c'])],)

TheExtension class (actually, the underlying extension-buildingmachinery implemented by thebuild_ext command) supports a great dealof flexibility in describing Python extensions, which is explained in thefollowing sections.

Extension('foo',['src/foo1.c','src/foo2.c'])

Extension('pkg.foo',['src/foo1.c','src/foo2.c'])

setup(...,ext_package='pkg',ext_modules=[Extension('foo',['foo.c']),Extension('subpkg.bar',['bar.c'])],)

setup(...,ext_modules=[Extension('_foo',['foo.i'],swig_opts=['-modern','-I../include'])],py_modules=['foo'],)

>pythonsetup.pybuild_ext--swig-opts="-modern -I../include"

Extension('foo',['foo.c'],include_dirs=['include'])

Extension('foo',['foo.c'],include_dirs=['/usr/include/X11'])

#include <X11/Xlib.h>

#include <Numerical/arrayobject.h>

fromdistutils.sysconfigimportget_python_incincdir=os.path.join(get_python_inc(plat_specific=1),'Numerical')setup(...,Extension(...,include_dirs=[incdir]),)

Extension(...,define_macros=[('NDEBUG','1'),('HAVE_STRFTIME',None)],undef_macros=['HAVE_FOO','HAVE_BAR'])

#define NDEBUG 1#define HAVE_STRFTIME#undef HAVE_FOO#undef HAVE_BAR

Extension(...,libraries=['gdbm','readline'])

Extension(...,library_dirs=['/usr/X11R6/lib'],libraries=['X11','Xt'])

2.4.Relationships between Distributions and Packages¶

A distribution may relate to packages in three specific ways:

1. It can require packages or modules.

2. It can provide packages or modules.

3. It can obsolete packages or modules.

These relationships can be specified using keyword arguments to thedistutils.core.setup() function.

Dependencies on other Python modules and packages can be specified by supplyingtherequires keyword argument tosetup(). The value must be a list ofstrings. Each string specifies a package that is required, and optionally whatversions are sufficient.

To specify that any version of a module or package is required, the stringshould consist entirely of the module or package name. Examples include'mymodule' and'xml.parsers.expat'.

If specific versions are required, a sequence of qualifiers can be supplied inparentheses. Each qualifier may consist of a comparison operator and a versionnumber. The accepted comparison operators are:

<>==<=>=!=

These can be combined by using multiple qualifiers separated by commas (andoptional whitespace). In this case, all of the qualifiers must be matched; alogical AND is used to combine the evaluations.

Let’s look at a bunch of examples:

Now that we can specify dependencies, we also need to be able to specify what weprovide that other distributions can require. This is done using theprovideskeyword argument tosetup(). The value for this keyword is a list ofstrings, each of which names a Python module or package, and optionallyidentifies the version. If the version is not specified, it is assumed to matchthat of the distribution.

Some examples:

A package can declare that it obsoletes other packages using theobsoleteskeyword argument. The value for this is similar to that of therequireskeyword: a list of strings giving module or package specifiers. Each specifierconsists of a module or package name optionally followed by one or more versionqualifiers. Version qualifiers are given in parentheses after the module orpackage name.

The versions identified by the qualifiers are those that are obsoleted by thedistribution being described. If no qualifiers are given, all versions of thenamed module or package are understood to be obsoleted.

2.5.Installing Scripts¶

So far we have been dealing with pure and non-pure Python modules, which areusually not run by themselves but imported by scripts.

Scripts are files containing Python source code, intended to be started from thecommand line. Scripts don’t require Distutils to do anything very complicated.The only clever feature is that if the first line of the script starts with#! and contains the word “python”, the Distutils will adjust the first lineto refer to the current interpreter location. By default, it is replaced withthe current interpreter location. The--executable (or-e)option will allow the interpreter path to be explicitly overridden.

Thescripts option simply is a list of files to be handled in thisway. From the PyXML setup script:

setup(...,scripts=['scripts/xmlproc_parse','scripts/xmlproc_val'])

2.6.Installing Package Data¶

Often, additional files need to be installed into a package. These files areoften data that’s closely related to the package’s implementation, or text filescontaining documentation that might be of interest to programmers using thepackage. These files are calledpackage data.

Package data can be added to packages using thepackage_data keywordargument to thesetup() function. The value must be a mapping frompackage name to a list of relative path names that should be copied into thepackage. The paths are interpreted as relative to the directory containing thepackage (information from thepackage_dir mapping is used if appropriate);that is, the files are expected to be part of the package in the sourcedirectories. They may contain glob patterns as well.

The path names may contain directory portions; any necessary directories will becreated in the installation.

For example, if a package should contain a subdirectory with several data files,the files can be arranged like this in the source tree:

setup.pysrc/mypkg/__init__.pymodule.pydata/tables.datspoons.datforks.dat

The corresponding call tosetup() might be:

setup(...,packages=['mypkg'],package_dir={'mypkg':'src/mypkg'},package_data={'mypkg':['data/*.dat']},)

2.7.Installing Additional Files¶

Thedata_files option can be used to specify additional files neededby the module distribution: configuration files, message catalogs, data files,anything which doesn’t fit in the previous categories.

data_files specifies a sequence of (directory,files) pairs in thefollowing way:

setup(...,data_files=[('bitmaps',['bm/b1.gif','bm/b2.gif']),('config',['cfg/data.cfg'])],)

Each (directory,files) pair in the sequence specifies the installationdirectory and the files to install there.

Each file name infiles is interpreted relative to thesetup.pyscript at the top of the package source distribution. Note that you canspecify the directory where the data files will be installed, but you cannotrename the data files themselves.

Thedirectory should be a relative path. It is interpreted relative to theinstallation prefix (Python’ssys.prefix for system installations;site.USER_BASE for user installations). Distutils allowsdirectory to bean absolute installation path, but this is discouraged since it isincompatible with the wheel packaging format. No directory information fromfiles is used to determine the final location of the installed file; onlythe name of the file is used.

You can specify thedata_files options as a simple sequence of fileswithout specifying a target directory, but this is not recommended, and theinstall command will print a warning in this case. To install datafiles directly in the target directory, an empty string should be given as thedirectory.

2.8.Additional meta-data¶

The setup script may include additional meta-data beyond the name and version.This information includes:

Notes:

1. These fields are required.

2. It is recommended that versions take the formmajor.minor[.patch[.sub]].

3. Either the author or the maintainer must be identified. If maintainer isprovided, distutils lists it as the author inPKG-INFO.

4. Thelong_description field is used by PyPI when you publish a package,to build its project page.

5. Thelicense field is a text indicating the license covering thepackage where the license is not a selection from the “License” Troveclassifiers. See theClassifier field. Notice thatthere’s alicence distribution option which is deprecated but stillacts as an alias forlicense.

6. This field must be a list.

7. The valid classifiers are listed onPyPI.

8. To preserve backward compatibility, this field also accepts a string. Ifyou pass a comma-separated string'foo,bar', it will be converted to['foo','bar'], Otherwise, it will be converted to a list of onestring.

‘short string’

A single line of text, not more than 200 characters.

‘long string’

Multiple lines of plain text in reStructuredText format (seehttps://docutils.sourceforge.io/).

‘list of strings’

See below.

Encoding the version information is an art in itself. Python packages generallyadhere to the version formatmajor.minor[.patch][sub]. The major number is 0for initial, experimental releases of software. It is incremented for releasesthat represent major milestones in a package. The minor number is incrementedwhen important new features are added to the package. The patch numberincrements when bug-fix releases are made. Additional trailing versioninformation is sometimes used to indicate sub-releases. These are“a1,a2,…,aN” (for alpha releases, where functionality and API may change),“b1,b2,…,bN” (for beta releases, which only fix bugs) and “pr1,pr2,…,prN”(for final pre-release release testing). Some examples:

0.1.0

the first, experimental release of a package

1.0.1a2

the second alpha release of the first patch version of 1.0

classifiers must be specified in a list:

setup(...,classifiers=['Development Status :: 4 - Beta','Environment :: Console','Environment :: Web Environment','Intended Audience :: End Users/Desktop','Intended Audience :: Developers','Intended Audience :: System Administrators','License :: OSI Approved :: Python Software Foundation License','Operating System :: MacOS :: MacOS X','Operating System :: Microsoft :: Windows','Operating System :: POSIX','Programming Language :: Python','Topic :: Communications :: Email','Topic :: Office/Business','Topic :: Software Development :: Bug Tracking',],)

2.9.Debugging the setup script¶

Sometimes things go wrong, and the setup script doesn’t do what the developerwants.

Distutils catches any exceptions when running the setup script, and print asimple error message before the script is terminated. The motivation for thisbehaviour is to not confuse administrators who don’t know much about Python andare trying to install a package. If they get a big long traceback from deepinside the guts of Distutils, they may think the package or the Pythoninstallation is broken because they don’t read all the way down to the bottomand see that it’s a permission problem.

On the other hand, this doesn’t help the developer to find the cause of thefailure. For this purpose, theDISTUTILS_DEBUG environment variable can be setto anything except an empty string, and distutils will now print detailedinformation about what it is doing, dump the full traceback when an exceptionoccurs, and print the whole command line when an external program (like a Ccompiler) fails.