src-layout¶

The project should contain asrc directory under the project root andall modules and packages meant for distribution are placed inside thisdirectory:

project_root_directory├── pyproject.toml  # AND/OR setup.cfg, setup.py├── ...└── src/    └── mypkg/        ├── __init__.py        ├── ...        ├── module.py        ├── subpkg1/        │   ├── __init__.py        │   ├── ...        │   └── module1.py        └── subpkg2/            ├── __init__.py            ├── ...            └── module2.py

This layout is very handy when you wish to use automatic discovery,since you don’t have to worry about other Python files or folders in yourproject root being distributed by mistake. In some circumstances it can bealso less error-prone for testing or when usingPEP 420-style packages.On the other hand you cannot rely on the implicitPYTHONPATH=. to fireup the Python REPL and play with your package (you will need aneditable install to be able to do that).

flat-layout¶

(also known as “adhoc”)

The package folder(s) are placed directly under the project root:

project_root_directory├── pyproject.toml  # AND/OR setup.cfg, setup.py├── ...└── mypkg/    ├── __init__.py    ├── ...    ├── module.py    ├── subpkg1/    │   ├── __init__.py    │   ├── ...    │   └── module1.py    └── subpkg2/        ├── __init__.py        ├── ...        └── module2.py

This layout is very practical for using the REPL, but in some situationsit can be more error-prone (e.g. during tests or if you have a bunchof folders or Python files hanging around your project root).

To avoid confusion, file and folder names that are used by popular tools (orthat correspond to well-known conventions, such as distributing documentationalongside the project code) are automatically filtered out in the case offlat-layout:

FlatLayoutPackageFinder.DEFAULT_EXCLUDE:ClassVar[tuple[str,...]]=('ci','ci.*','bin','bin.*','debian','debian.*','doc','doc.*','docs','docs.*','documentation','documentation.*','manpages','manpages.*','news','news.*','newsfragments','newsfragments.*','changelog','changelog.*','test','test.*','tests','tests.*','unit_test','unit_test.*','unit_tests','unit_tests.*','example','example.*','examples','examples.*','scripts','scripts.*','tools','tools.*','util','util.*','utils','utils.*','python','python.*','build','build.*','dist','dist.*','venv','venv.*','env','env.*','requirements','requirements.*','tasks','tasks.*','fabfile','fabfile.*','site_scons','site_scons.*','benchmark','benchmark.*','benchmarks','benchmarks.*','exercise','exercise.*','exercises','exercises.*','htmlcov','htmlcov.*','[._]*','[._]*.*')¶

Reserved package names

FlatLayoutModuleFinder.DEFAULT_EXCLUDE:ClassVar[tuple[str,...]]=('setup','conftest','test','tests','example','examples','build','toxfile','noxfile','pavement','dodo','tasks','fabfile','[Ss][Cc]onstruct','conanfile','manage','benchmark','benchmarks','exercise','exercises','[._]*')¶

Reserved top-level module names

There is also a handy variation of theflat-layout for utilities/librariesthat can be implemented with a single Python file:

project_root_directory├── pyproject.toml  # AND/OR setup.cfg, setup.py├── ...└── single_file_lib.py

Finding simple packages¶

Let’s start with the first tool.find: (find_packages()) takes a sourcedirectory and two lists of package name patterns to exclude and include, andthen returns a list ofstr representing the packages it could find. To useit, consider the following directory:

mypkg├── pyproject.toml  # AND/OR setup.cfg, setup.py└── src    ├── pkg1    │   └── __init__.py    ├── pkg2    │   └── __init__.py    ├── additional    │   └── __init__.py    └── pkg        └── namespace            └── __init__.py

To have setuptools to automatically include packages foundinsrc that start with the namepkg and notadditional:

[options]packages=find:package_dir==src[options.packages.find]where=srcinclude=pkg*# alternatively: `exclude = additional*`

setup(# ...packages=find_packages(where='src',include=['pkg*'],# alternatively: `exclude=['additional*']`),package_dir={"":"src"}# ...)

[tool.setuptools.packages.find]where=["src"]include=["pkg*"]# alternatively: `exclude = ["additional*"]`namespaces=false

Finding namespace packages¶

setuptools providesfind_namespace: (find_namespace_packages())which behaves similarly tofind: but works with namespace packages.

Before diving in, it is important to have a good understanding of whatnamespace packages are. Here is a quick recap.

When you have two packages organized as follows:

/Users/Desktop/timmins/foo/__init__.py/Library/timmins/bar/__init__.py

If bothDesktop andLibrary are on yourPYTHONPATH, then anamespace package calledtimmins will be created automatically for you whenyou invoke the import mechanism, allowing you to accomplish the following:

>>>importtimmins.foo>>>importtimmins.bar

as if there is only onetimmins on your system. The two packages can thenbe distributed separately and installed individually without affecting theother one.

Now, suppose you decide to package thefoo part for distribution and startby creating a project directory organized as follows:

foo├── pyproject.toml  # AND/OR setup.cfg, setup.py└── src    └── timmins        └── foo            └── __init__.py

If you want thetimmins.foo to be automatically included in thedistribution, then you will need to specify:

[options]package_dir==srcpackages=find_namespace:[options.packages.find]where=src

setup(# ...packages=find_namespace_packages(where='src'),package_dir={"":"src"}# ...)

[tool.setuptools.packages.find]where=["src"]

After installing the package distribution,timmins.foo would becomeavailable to your interpreter.

foo├── docs│   └── conf.py├── timmins│   └── foo│       └── __init__.py└── tests    └── tests_foo        └── __init__.py

tartfdist/*.tar.gzunzip-ldist/*.whl

pkg_resource style namespace package¶

This is the methodsetuptools directly supports. Starting with the samelayout, there are two pieces you need to add to it. First, an__init__.pyfile directly under your namespace package directory that contains thefollowing:

__import__("pkg_resources").declare_namespace(__name__)

And thenamespace_packages keyword in yoursetup.cfg orsetup.py:

[options]namespace_packages=timmins

setup(# ...namespace_packages=['timmins'])

And your directory should look like this

foo├──pyproject.toml# AND/OR setup.cfg, setup.py└──src└──timmins├──__init__.py└──foo└──__init__.py

Repeat the same for other packages and you can achieve the same result asthe previous section.

pkgutil style namespace package¶

This method is almost identical to thepkg_resource except that thenamespace_packages declaration is omitted and the__init__.pyfile contains the following:

__path__=__import__('pkgutil').extend_path(__path__,__name__)

The project layout remains the same andpyproject.toml/setup.cfg remains the same.

https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure

https://blog.ionelmc.ro/2017/09/25/rehashing-the-src-layout/