pypiserver - minimal PyPI server for use with pip/easy_install

Tip

Reach out inDiscussions,or chat with us onZulip

Note

The official software poweringPyPI isWarehouse. However,Warehouseis fairly specialized to bepypi.org's own software, and should notbe used in other contexts. In particular, it does not officially supportbeing used as a custom package index by users wishing to serve their ownpackages.

pypiserver

Quickstart Installation and Usage

Important

pypiserver works with Python 3.8+ and PyPy3.

Warning

Older Python versions may still work, but they are not tested.

For legacy Python versions, usepypiserver-1.x series.
Note that these are not officially supported, and will not receive bugfixes or new features.

Tip

The commands below work on a unix-like operating system with a posix shell.
The'~' character expands to user's home directory.

If you're using Windows, you'll have to use their "Windows counterparts".
The same is true for the rest of this documentation.

More details about pypi server run

usage: pypi-server run [-h] [-v] [--log-file FILE] [--log-stream STREAM]                       [--log-frmt FORMAT] [--hash-algo HASH_ALGO]                       [--backend {auto,simple-dir,cached-dir}] [--version]                       [-p PORT] [-i HOST] [-a AUTHENTICATE]                       [-P PASSWORD_FILE] [--disable-fallback]                       [--fallback-url FALLBACK_URL]                       [--health-endpoint HEALTH_ENDPOINT] [--server METHOD]                       [-o] [--welcome HTML_FILE] [--cache-control AGE]                       [--log-req-frmt FORMAT] [--log-res-frmt FORMAT]                       [--log-err-frmt FORMAT]                       [--server-base-url SERVER_BASE_URL]                       [package_directory ...]positional arguments:  package_directory     The directory from which to serve packages.options:  -h, --help            show this help message and exit  -v, --verbose         Enable verbose logging; repeat for more verbosity.  --log-file FILE       Write logging info into this FILE, as well as to                        stdout or stderr, if configured.  --log-stream STREAM   Log messages to the specified STREAM. Valid values are                        stdout, stderr, and none  --log-frmt FORMAT     The logging format-string.  (see `logging.LogRecord`                        class from standard python library)  --hash-algo HASH_ALGO                        Any `hashlib` available algorithm to use for                        generating fragments on package links. Can be disabled                        with one of (0, no, off, false).  --backend {auto,simple-dir,cached-dir}                        A backend implementation. Keep the default 'auto' to                        automatically determine whether to activate caching or                        not  --version             show program's version number and exit  -p PORT, --port PORT  Listen on port PORT (default: 8080)  -i HOST, -H HOST, --interface HOST, --host HOST                        Listen on interface INTERFACE (default: 0.0.0.0)  -a AUTHENTICATE, --authenticate AUTHENTICATE                        Comma-separated list of (case-insensitive) actions to                        authenticate (options: download, list, update;                        default: update).                                                  Any actions not specified are not authenticated, so                         to authenticate downloads and updates, but allow                         unauthenticated viewing of the package list, you would                         use:                                                   pypi-server -a 'download, update' -P                          ./my_passwords.htaccess                                                 To disable authentication, use:                                                   pypi-server -a . -P .                                                 See the `-P` option for configuring users and                        passwords.                                                 Note that when uploads are not protected, the                        `register` command is not necessary, but `~/.pypirc`                        still needs username and password fields, even if                        bogus.  -P PASSWORD_FILE, --passwords PASSWORD_FILE                        Use an apache htpasswd file PASSWORD_FILE to set                        usernames and passwords for authentication.                                                 To allow unauthorized access, use:                                                   pypi-server -a . -P .                           --disable-fallback    Disable the default redirect to PyPI for packages not                        found in the local index.  --fallback-url FALLBACK_URL                        Redirect to FALLBACK_URL for packages not found in the                        local index.  --health-endpoint HEALTH_ENDPOINT                        Configure a custom liveness endpoint. It always                        returns 200 Ok if the service is up. Otherwise, it                        means that the service is not responsive.  --server METHOD       Use METHOD to run the server. Valid values include                        paste, cherrypy, twisted, gunicorn, gevent, wsgiref,                        and auto. The default is to use "auto", which chooses                        one of paste, cherrypy, twisted, or wsgiref.  -o, --overwrite       Allow overwriting existing package files during                        upload.  --welcome HTML_FILE   Use the contents of HTML_FILE as a custom welcome                        message on the home page.  --cache-control AGE   Add "Cache-Control: max-age=AGE" header to package                        downloads. Pip 6+ requires this for caching.AGE is                        specified in seconds.  --log-req-frmt FORMAT                        A format-string selecting Http-Request properties to                        log; set to '%s' to see them all.  --log-res-frmt FORMAT                        A format-string selecting Http-Response properties to                        log; set to '%s' to see them all.  --log-err-frmt FORMAT                        A format-string selecting Http-Error properties to                        log; set to '%s' to see them all.  --server-base-url SERVER_BASE_URL                        Serve all routes under SERVER_BASE_URL prefix                        (default: {DEFAULTS.SERVER_BASE_URL})

More details about pypi-server update

usage: pypi-server update [-h] [-v] [--log-file FILE] [--log-stream STREAM]                          [--log-frmt FORMAT] [--hash-algo HASH_ALGO]                          [--backend {auto,simple-dir,cached-dir}] [--version]                          [-x] [-d DOWNLOAD_DIRECTORY] [-u]                          [--blacklist-file IGNORELIST_FILE]                          [package_directory ...]positional arguments:  package_directory     The directory from which to serve packages.options:  -h, --help            show this help message and exit  -v, --verbose         Enable verbose logging; repeat for more verbosity.  --log-file FILE       Write logging info into this FILE, as well as to                        stdout or stderr, if configured.  --log-stream STREAM   Log messages to the specified STREAM. Valid values are                        stdout, stderr, and none  --log-frmt FORMAT     The logging format-string. (see `logging.LogRecord`                        class from standard python library)  --hash-algo HASH_ALGO                        Any `hashlib` available algorithm to use for                        generating fragments on package links. Can be disabled                        with one of (0, no, off, false).  --backend {auto,simple-dir,cached-dir}                        A backend implementation. Keep the default 'auto' to                        automatically determine whether to activate caching or                        not  --version             show program's version number and exit  -x, --execute         Execute the pip commands rather than printing to                        stdout  -d DOWNLOAD_DIRECTORY, --download-directory DOWNLOAD_DIRECTORY                        Specify a directory where packages updates will be                        downloaded. The default behavior is to use the                        directory which contains the package being updated.  -u, --allow-unstable  Allow updating to unstable versions (alpha, beta, rc,                        dev, etc.)  --blacklist-file IGNORELIST_FILE, --ignorelist-file IGNORELIST_FILE                        Don't update packages listed in this file (one package                        name per line, without versions, '#' comments                        honored). This can be useful if you upload private                        packages into pypiserver, but also keep a mirror of                        public packages that you regularly update. Attempting                        to pull an update of a private package from `pypi.org`                        might pose a security risk - e.g. a malicious user                        might publish a higher version of the private package,                        containing arbitrary code.

Client-Side Configurations

Tip

Sincepypiserver redirectspip/easy_install to thepypi.org indexif it doesn't have a requested package, it is a good idea to configure them toalways use your local pypi index.

Configuring pip

export PIP_EXTRA_INDEX_URL=http://localhost:8080/simple/

[global]extra-index-url = http://localhost:8080/simple/

Note

If you have installedpypiserver on a remote url withouthttpsyou will receive an "untrusted" warning frompip, urging you to appendthe--trusted-host option. You can also include this option permanentlyin your configuration-files or environment variables.

Configuring easy_install

[easy_install]index_url = http://localhost:8080/simple/

Uploading Packages Remotely

Important

We strongly advise topassword-protect your uploads!

Apache Like Authentication (htpasswd)

Tip

Read thisSOquestion for runninghtpasswd cmd underWindowsor if you have bogus passwords that you don't care because they are foran internal service (which is still "bad", from a security perspective...)you may use thispublic service

Tip

When accessing pypiserver via the api, alternate authenticationmethods are available via theauther config flag. Any callablereturning a boolean can be passed through to the pypiserver config inorder to provide custom authentication. For example, to configurepypiserver to authenticate using thepython-pam

importpampypiserver.default_config(auther=pam.authenticate)

Upload with setuptools

Upload with twine

Using the Docker Image

docker run pypiserver/pypiserver:latest run

docker run -p 80:8080 pypiserver/pypiserver:latest run

docker run -p 80:8080 -v~/packages:/data/packages pypiserver/pypiserver:latest run

docker run -p 80:8080 -v~/.htpasswd:/data/.htpasswd pypiserver/pypiserver:latest run -P .htpasswd packages

Alternative Installation Methods

# VERSION-CHECK: Fails if not installed.pypi-server --version# UNINSTALL: Invoke again until it fails.pip uninstall pypiserver

Installing the Very Latest Version

pip install pypiserver --pre -I

pip install git+git://github.com/pypiserver/pypiserver.git

Recipes

Managing the Package Directory

$ ./pypi-server update checking 106 packagesfor newer version.........u.e...........e..u..................e..............................e.............................no releases found on pypifor PyXML, Pymacs, mercurial, setuptools# update raven from 1.4.3 to 1.4.4pip -q install --no-deps  --extra-index-url https://pypi.org/simple/ -d /home/ralf/packages/mirror raven==1.4.4# update greenlet from 0.3.3 to 0.3.4pip -q install --no-deps  --extra-index-url https://pypi.org/simple/ -d /home/ralf/packages/mirror greenlet==0.3.4

Serving Thousands of Packages

Important

By default,pypiserver scans the entire packages directory each time anincoming HTTP request occurs. This isn't a problem for a small number ofpackages, but causes noticeable slow-downs when serving thousands of packages.

pip install pypiserver[cache]

proxy_cache_path /data/nginx/cache                  levels=1:2                  keys_zone=pypiserver_cache:10m                  max_size=10g                  inactive=60m                  use_temp_path=off;server{    # ...location /{proxy_cache pypiserver_cache;proxy_passhttp://localhost:8080;}}

Tip

Using webserver caching is especially helpful if you have high requestvolume. Using nginx caching, a real-world pypiserver installation wasable to easily support over 1000 package downloads/min at peak load.

Managing Automated Startup

Running As a systemd Service

[Unit]Description=A minimal PyPI server for use with pip/easy_install.After=network.target[Service]Type=simple# systemd requires absolute path here too.PIDFile=/var/run/pypiserver.pidUser=www-dataGroup=www-dataExecStart=/usr/local/bin/pypi-server run -p 8080 -a update,download --log-file /var/log/pypiserver.log -P /etc/nginx/.htpasswd /var/www/pypiExecStop=/bin/kill -TERM $MAINPIDExecReload=/bin/kill -HUP $MAINPIDRestart=alwaysWorkingDirectory=/var/www/pypiTimeoutStartSec=3RestartSec=5[Install]WantedBy=multi-user.target

Launching through supervisor

[program:pypi]command=/home/pypi/pypi-venv/bin/pypi-server run -p 7001 -P /home/pypi/.htpasswd /home/pypi/packagesdirectory=/home/pypiuser=pypiautostart=trueautorestart=truestderr_logfile=/var/log/pypiserver.err.logstdout_logfile=/var/log/pypiserver.out.log

Running As a service with NSSM

pypi-server run -p 8080 C:\Path\To\Packages&

Tip

Test the batch file by running it first before creating the service.Make sure you can access the server remotely, and install packages. If you can,proceed, if not troubleshoot until you can. This will ensure you know the serverworks, before adding NSSM into the mix.

nssm install pypiserver

Path: C:\Path\To\start_pypiserver.batStartup directory: Auto generates when selecting pathService name: pypiserver

nssm start pypiserver

Tip

Other useful commands

nssm --helpnssm stop<servicename>nssm restart<servicename>nssm status<servicename>

For detailed information please visithttps://nssm.cc

Using a Different WSGI Server

Apache

Note

If you choose instead to go withmod_proxy, mind that you may bump into problemswith the prefix-path (see#155).

Note

For security reasons, notice that theDirectory directive grants accessto a directory holding thewsgi start-up script, alone; nothing else.

Note

To enable HTTPS support on Apache, configure the directive that contains theWSGI configuration to use SSL.

gunicorn

gunicorn -w4'pypiserver:app(root="/home/ralf/packages")'

gunicorn -w4'pypiserver:app(root=["/home/ralf/packages", "/home/ralf/experimental"])'

paste

[composite:main]use = egg:Paste#urlmap/unstable/ = unstable/ = stable[app:stable]use = egg:pypiserver#mainroot = ~/stable-packages[app:unstable]use = egg:pypiserver#mainroot = ~/stable-packages    ~/unstable-packages[server:main]use = egg:gunicorn#mainhost = 0.0.0.0port = 9000workers = 5accesslog = -

Note

You need to install some more dependencies for this to work, like::

pip install paste pastedeploy gunicorn pypiserver

The server can then start with

gunicorn_paster paste.ini

Behind a Reverse Proxy

Nginx

upstream pypi{server              pypiserver.example.com:12345 fail_timeout=0;}server{server_name         myproxy.example.com;location /{proxy_set_header  Host$host:$server_port;proxy_set_header  X-Forwarded-Proto$scheme;proxy_set_header  X-Real-IP$remote_addr;proxy_passhttp://pypi;}}

upstream pypi{server              localhost:8000;}server{location /pypi/{proxy_set_header  X-Forwarded-Host$host:$server_port/pypi;proxy_set_header  X-Forwarded-Proto$scheme;proxy_set_header  X-Forwarded-For$proxy_add_x_forwarded_for;proxy_set_header  X-Real-IP$remote_addr;proxy_passhttp://pypi;}}

Supporting HTTPS

upstream pypi{server               localhost:8000;}server{listen80 default_server;server_name         _;return301 https://$host$request_uri;}server{listen443ssl;server_name         pypiserver.example.com;ssl_certificate     /etc/star.example.com.crt;ssl_certificate_key /etc/star.example.com.key;ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;ssl_ciphers         HIGH:!aNULL:!MD5;location /{proxy_set_header  Host$host:$server_port;proxy_set_header  X-Forwarded-Proto$scheme;proxy_set_header  X-Real-IP$remote_addr;proxy_passhttp://pypi;}}

Tip

Please seenginx's HTTPS docs for more details.

Getting and keeping your certificates up-to-date can be simplified using,for example, usingcertbot and letsencrypt.

Traefik

Utilizing the API

Using Ad-Hoc Authentication Providers

Note

Thepython-pam module, requiresread access to/etc/shadow file;you may add the user under whichpypiserver runs into theshadowgroup, with a command like this:sudo usermod -a -G shadow pypy-user.

Use with MicroPython

cd micropythonports/unix/micropython -m tools.upip install -i http://my-server:8080 -p /tmp/mymodules micropython-foobar

importnetworkimportupipsta_if=network.WLAN(network.STA_IF)sta_if.active(True)sta_if.connect('<your ESSID>','<your password>')upip.index_urls= ["http://my-server:8080"]upip.install("micropython-foobar")

Custom Health Check Endpoint

Configure a custom health endpoint by CLI arguments

pypi-server run --health-endpoint /action/health

Configure a custom health endpoint by script

importpypiserverfrompypiserverimportbottle_wrapperasbottleapp=pypiserver.app(root="./packages",health_endpoint="/action/health")bottle.run(app=app,host="0.0.0.0",port=8080,server="auto")

Exposingpypi-server on a URL with a custom prefix

Sources

git clone https://github.com/pypiserver/pypiserver.gitcd pypiserver

git pull

Known Limitations

Important

pypiserver does not implement the full API as seen onPyPI.It implements just enough to makeeasy_install,pip install, andsearch work.

Similar Projects

Unmaintained or archived

Related Software

Licensing