Repository files navigation

A caching proxy for Docker; allows centralised management of (multiple) registries and their authentication; caches images fromany registry.Caches the potentially huge blob/layer requests (for bandwidth/time savings), and optionally caches manifest requests ("pulls") to avoid rate-limiting.

Docker, Inc has announced a2nd apocalypse for 1st of March'25 (it hasalready been pushed back to April).This has caused a new surge of interest in this project; in response I've updated all dependencies to the latest versions,added aTest matrix, merged some pull requests (includingDISABLE_IPV6=true, which was a long-standing request), and updated the documentation.

Many thanks to all the contributors over the years; I've no intention of abandoning this project -- please keep sending and updating your PRs.

Starting November 2nd, 2020, DockerHub willsupposedlystartrate-limiting pulls,also known as theDocker Apocalypse.The main symptom isError response from daemon: toomanyrequests: Too Many Requests. Please see https://docs.docker.com/docker-hub/download-rate-limit/ during pulls.Many unknowing Kubernetes clusters will hit the limit, and struggle to configureimagePullSecrets andimagePullPolicy.

This proxy can be configured with the env varENABLE_MANIFEST_CACHE=true which providesconfigurable caching of the manifest requests that DockerHub throttles. You can then fine-tune other parameters to your needs.Together with the possibility to centrally inject authentication (since 0.3x), this is probably one of the best ways to bring relief to your distressed cluster, while at the same time saving lots of bandwidth and time.

Note: enabling manifest caching, in its default config, effectively makes some tagsimmutable. Use with care. The configuration ENVs are explained in theDockerfile, relevant parts included below.

# Manifest caching tiers. Disabled by default, to mimick 0.4/0.5 behaviour.# Setting it to true enables the processing of the ENVs below.# Once enabled, it is valid for all registries, not only DockerHub.# The envs *_REGEX represent a regex fragment, check entrypoint.sh to understand how they're used (nginx ~ location, PCRE syntax).ENV ENABLE_MANIFEST_CACHE="false"# 'Primary' tier defaults to 10m cache for frequently used/abused tags.# - People publishing to production via :latest (argh) will want to include that in the regex# - Heavy pullers who are being ratelimited but don't mind getting outdated manifests should (also) increase the cache time hereENV MANIFEST_CACHE_PRIMARY_REGEX="(stable|nightly|production|test)"ENV MANIFEST_CACHE_PRIMARY_TIME="10m"# 'Secondary' tier defaults any tag that has 3 digits or dots, in the hopes of matching most explicitly-versioned tags.# It caches for 60d, which is also the cache time for the large binary blobs to which the manifests refer.# That makes them effectively immutable. Make sure you're not affected; tighten this regex or widen the primary tier.ENV MANIFEST_CACHE_SECONDARY_REGEX="(.*)(\d|\.)+(.*)(\d|\.)+(.*)(\d|\.)+"ENV MANIFEST_CACHE_SECONDARY_TIME="60d"# The default cache duration for manifests that don't match either the primary or secondary tiers above.# In the default config, :latest and other frequently-used tags will get this value.ENV MANIFEST_CACHE_DEFAULT_TIME="1h"

Essentially, it's aman in the middle: an intercepting proxy based onnginx, to which all docker traffic is directed using theHTTPS_PROXY mechanism and injected CA root certificates.

The main feature is Docker layer/image caching, including layers served from S3, Google Storage, etc.

As a bonus it allows for centralized management of Docker registry credentials, which can in itself be the main feature, eg in Kubernetes environments.

You configure the Docker clients (err... Kubernetes Nodes?) once, and then all configuration is done on the proxy --for this to work it requires inserting a root CA certificate into system trusted root certs.

• :latest and:latest-debug Docker tag is unstable, built from master, and amd64-only
• Production/stable is0.6.5, see0.6.5 tag on Github - this image is multi-arch amd64/arm64

• DockerHub image is atrpardini/docker-registry-proxy:<version>
• GitHub image is atghcr.io/rpardini/docker-registry-proxy:<version>
• Since 0.5.x, they both carry the same images
• This can be useful if you're already hitting DockerHub's rate limits and can't pull the proxy from DockerHub

• Run the proxy on a host close (network-wise: high bandwidth, same-VPC, etc) to the Docker clients
• Expose port 3128 to the network
• Map volume/docker_mirror_cache for up toCACHE_MAX_SIZE (32gb by default) of cached images across all cached registries
• Map volume/ca, the proxy will store the CA certificate here across restarts.Important this is security sensitive.
• EnvALLOW_PUSH : This bypasses the proxy when pushing, default to false - if kept to false, pushing will not work. For more info see thiscommit.
• EnvCACHE_MAX_SIZE (default32g): set the max size to be used for caching local Docker image layers. UseNginx sizes.
• EnvENABLE_MANIFEST_CACHE, see the section on pull rate limiting.
• EnvREGISTRIES: space separated list of registries to cache; no need to include DockerHub, its already done internally.
• EnvAUTH_REGISTRIES: space separated list ofhostname:username:password authentication info.
  • hostnames listed here should be listed in the REGISTRIES environment as well, so they can be intercepted.
• EnvAUTH_REGISTRIES_DELIMITER to change the separator between authentication info. By default, a space: "". If you use keys that contain spaces (as with Google Cloud Registry), you should update this variable, e.g. setting it toAUTH_REGISTRIES_DELIMITER=";;;". In that case,AUTH_REGISTRIES could contain something likeregistry1.com:user1:pass1;;;registry2.com:user2:pass2.
• EnvAUTH_REGISTRY_DELIMITER to change the separator between authentication infoparts. By default, a colon: ":". If you use keys that contain single colons, you should update this variable, e.g. setting it toAUTH_REGISTRY_DELIMITER=":::". In that case,AUTH_REGISTRIES could contain something likeregistry1.com:::user1:::pass1 registry2.com:::user2:::pass2.
• EnvPROXY_REQUEST_BUFFERING: If push is allowed, buffering requests can cause issues on slow upstreams. If you have trouble pushing, set this tofalse first, then fix remaining timeouts. Default istrue to not change default behavior.
• Timeouts ENVS - all of them can pe specified to control different timeouts, and if not set, the defaults will be the ones fromDockerfile. The directives will be added intohttp block.:
  • SEND_TIMEOUT : seesend_timeout
  • CLIENT_BODY_TIMEOUT : seeclient_body_timeout
  • CLIENT_HEADER_TIMEOUT : seeclient_header_timeout
  • KEEPALIVE_TIMEOUT : seekeepalive_timeout
  • PROXY_READ_TIMEOUT : seeproxy_read_timeout
  • PROXY_CONNECT_TIMEOUT : seeproxy_connect_timeout
  • PROXY_SEND_TIMEOUT : seeproxy_send_timeout
  • PROXY_CONNECT_READ_TIMEOUT : seeproxy_connect_read_timeout
  • PROXY_CONNECT_CONNECT_TIMEOUT : seeproxy_connect_connect_timeout
  • PROXY_CONNECT_SEND_TIMEOUT : seeproxy_connect_send_timeout
• EnvDISABLE_IPV6: If set totrue, prevents nginx from getting IPv6 addresses from the resolver, without needing acustom resolver config

docker run --rm --name docker_registry_proxy -it \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       rpardini/docker-registry-proxy:0.6.5

For Docker Hub authentication:

• hostname should beauth.docker.io
• username should NOT be an email, use the regular username

docker run --rm --name docker_registry_proxy -it \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       -e REGISTRIES="registry.k8s.io gcr.io quay.io your.own.registry another.public.registry" \       -e AUTH_REGISTRIES="auth.docker.io:dockerhub_username:dockerhub_password your.own.registry:username:password" \       rpardini/docker-registry-proxy:0.6.5

For regular registry auth (HTTP Basic), thehostname should be the registry itself... unless your registry uses a different auth server.

See the example above for DockerHub, adapt theyour.own.registry parts (in both ENVs).

This should work for quay.io also, but I have no way to test.

GitLab may use a different/separate domain to handle the authentication procedure.

Just like DockerHub usesauth.docker.io, GitLab uses its primary (git) domain for the authentication.

If you run GitLab ongit.example.com and its registry onreg.example.com, you need to include both inREGISTRIES and use the primary domain forAUTH_REGISTRIES.

For GitLab.com itself the authentication domain should begitlab.com.

docker run  --rm --name docker_registry_proxy -it \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       -e REGISTRIES="reg.example.com git.example.com" \       -e AUTH_REGISTRIES="git.example.com:USER:PASSWORD" \       rpardini/docker-registry-proxy:0.6.5

For Google Container Registry (GCR), username should be_json_key and the password should be the contents of the service account JSON.Check outGCR docs.

The service account key is in JSON format, it contains spaces ("") and colons (":").

To be able to use GCR you should setAUTH_REGISTRIES_DELIMITER to something different than space (e.g.AUTH_REGISTRIES_DELIMITER=";;;") andAUTH_REGISTRY_DELIMITER to something different than a single colon (e.g.AUTH_REGISTRY_DELIMITER=":::").

Example with GCR using credentials from a service account from a key fileservicekey.json:

docker run --rm --name docker_registry_proxy -it \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       -e REGISTRIES="registry.k8s.io gcr.io quay.io your.own.registry another.public.registry" \       -e AUTH_REGISTRIES_DELIMITER=";;;" \       -e AUTH_REGISTRY_DELIMITER=":::" \       -e AUTH_REGISTRIES="gcr.io:::_json_key:::$(cat servicekey.json);;;auth.docker.io:::dockerhub_username:::dockerhub_password" \       rpardini/docker-registry-proxy:0.6.5

For Google Artifact Registry (GAR), username should be_json_key and the password should be the contents of the service account JSON.Check outGAR docs.

The service account key is in JSON format, it contains spaces ("") and colons (":").

To be able to use GAR you should setAUTH_REGISTRIES_DELIMITER to something different than space (e.g.AUTH_REGISTRIES_DELIMITER=";;;") andAUTH_REGISTRY_DELIMITER to something different than a single colon (e.g.AUTH_REGISTRY_DELIMITER=":::").

GAR repositories have different domain names depending on the region in which they are hosted. SeparateREGISTRIES andAUTH_REGISTRIES entries must be defined for each region's domain name.us-east1-docker.pkg.dev andus-central1-docker.pkg.dev are used in the example below.

Example with GAR using credentials from a service account from a key fileservicekey.json:

docker run --rm --name docker_registry_proxy -it \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       -e REGISTRIES="us-east1-docker.pkg.dev us-central1-docker.pkg.dev" \       -e AUTH_REGISTRIES_DELIMITER=";;;" \       -e AUTH_REGISTRY_DELIMITER=":::" \       -e AUTH_REGISTRIES="us-east1-docker.pkg.dev:::_json_key:::$(cat servicekey.json);;;us-central1-docker.pkg.dev:::_json_key:::$(cat servicekey.json);;;auth.docker.io:::dockerhub_username:::dockerhub_password" \       rpardini/docker-registry-proxy:0.6.5

Kind is a tool for running local Kubernetes clusters using Docker container “nodes”.

Because cluster nodes are Docker containers, docker-registry-proxy needs to be in the same docker network.

Example joining thekind docker network and using hostnamedocker-registry-proxy as hostname :

docker run --rm --name docker_registry_proxy -it \       --net kind --hostname docker-registry-proxy \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       rpardini/docker-registry-proxy:0.6.5

Now deploy your Kind cluster and then automatically configure the nodes with the following script :

#!/bin/shKIND_NAME=${1-kind}SETUP_URL=http://docker-registry-proxy:3128/setup/systemdpids=""forNODEin$(kind get nodes --name"$KIND_NAME");do  dockerexec"$NODE" sh -c"\      curl$SETUP_URL\      | sed s/docker\.service/containerd\.service/g\      | sed '/Environment/ s/$/\"NO_PROXY=127.0.0.0\/8,10.0.0.0\/8,172.16.0.0\/12,192.168.0.0\/16\"/'\      | bash"& pids="$pids$!"# Configure every node in backgrounddonewait$pids# Wait for all configurations to end

K3d is similar to Kind but is based on k3s. In order to run with its registry you need to setup settings like shown below.

# docker-registry-proxydocker run -d --name registry-proxy --restart=always \-v /tmp/registry-proxy/mirror_cache:/docker_mirror_cache \-v /tmp/registry-proxy/certs:/ca \rpardini/docker-registry-proxy:0.6.5export PROXY_HOST=registry-proxyexport PROXY_PORT=3128export NOPROXY_LIST="localhost,127.0.0.1,0.0.0.0,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.local,.svc"cat<<EOF > /etc/k3d-proxy-config.yamlapiVersion: k3d.io/v1alpha3kind: Simplename: myclusterservers: 1agents: 0options:    k3d:       wait: true       timeout: "60s"    kubeconfig:       updateDefaultKubeconfig: true       switchCurrentContext: trueenv:  - envVar: HTTP_PROXY=http://$PROXY_HOST:$PROXY_PORT    nodeFilters:      - all  - envVar: HTTPS_PROXY=http://$PROXY_HOST:$PROXY_PORT    nodeFilters:      - all  - envVar: NO_PROXY='$NOPROXY_LIST'    nodeFilters:      - allvolumes:  - volume:$REGISTRY_DIR/docker_mirror_certs/ca.crt:/etc/ssl/certs/registry-proxy-ca.pem    nodeFilters:      - allEOFk3d cluster create --config /etc/k3d-proxy-config.yaml

Separate instructions for Mac clients available inthis dedicated Doc Desktop for Mac document.

Let's say you setup the proxy on host192.168.66.72, you can thencurl http://192.168.66.72:3128/ca.crt and get the proxy CA certificate.

On each Docker host that is to use the cache:

• Configure Docker proxy pointing to the caching server
• Add the caching server CA certificate to the list of system trusted roots.
• Restartdockerd

Do it all at once, tested on Ubuntu Xenial, Bionic, and Focal, all systemd based:

# Add environment vars pointing Docker to use the proxymkdir -p /etc/systemd/system/docker.service.dcat<<EOD > /etc/systemd/system/docker.service.d/http-proxy.conf[Service]Environment="HTTP_PROXY=http://192.168.66.72:3128/"Environment="HTTPS_PROXY=http://192.168.66.72:3128/"EOD### UBUNTU# Get the CA certificate from the proxy and make it a trusted root.curl http://192.168.66.72:3128/ca.crt> /usr/share/ca-certificates/docker_registry_proxy.crtecho"docker_registry_proxy.crt">> /etc/ca-certificates.confupdate-ca-certificates --fresh###### CENTOS# Get the CA certificate from the proxy and make it a trusted root.curl http://192.168.66.72:3128/ca.crt> /etc/pki/ca-trust/source/anchors/docker_registry_proxy.crtupdate-ca-trust#### Reload systemdsystemctl daemon-reload# Restart dockerdsystemctl restart docker.service

Cleardockerd of everything not currently running:docker system prune -a -fbeware

Then do, for example,docker pull registry.k8s.io/kube-proxy-amd64:v1.10.4 and watch the logs on the caching proxy, it should list a lot of MISSes.

Then, clean again, and pull again. You should see HITs! Success.

Do the same fordocker pull ubuntu and rejoice.

Test your own registry caching and authentication the same way; you don't needdocker login, or.docker/config.json anymore.

Since0.4 there is a separate-debug version of the image, which includesnginx-debug, and (since 0.5.x) has amitmproxy (actuallymitmweb) inserted after the CONNECT proxy but before the caching logic, and a secondmitmweb between the caching layer and DockerHub.This allows very in-depth debugging. Use sparingly, and definitely not in production.

docker run --rm --name docker_registry_proxy -it        -e DEBUG_NGINX=true -e DEBUG=true -e DEBUG_HUB=true -p 0.0.0.0:8081:8081 -p 0.0.0.0:8082:8082 \       -p 0.0.0.0:3128:3128 -e ENABLE_MANIFEST_CACHE=true \       -v$(pwd)/docker_mirror_cache:/docker_mirror_cache \       -v$(pwd)/docker_mirror_certs:/ca \       rpardini/docker-registry-proxy:0.6.5-debug

• DEBUG=true enables the mitmweb proxy between Docker clients and the caching layer, accessible on port 8081
• DEBUG_HUB=true enables the mitmweb proxy between the caching layer and DockerHub, accessible on port 8082 (since 0.5.x)
• DEBUG_NGINX=true enables nginx-debug and debug logging, which probably is too much. Seriously.

• If you authenticate to a private registry and pull through the proxy, those images will be served to any client that can reach the proxy, even without authentication.beware
• Repeat,this will make your private images very public if you're not careful.
• Currently you cannot push images while using the proxy which is a shame. PRs welcome.SEEALLOW_PUSH ENV FROM USAGE SECTION.
• Setting this on Linux is relatively easy.
  • On Mac follow the instructionshere.
  • On Windows follow the instructionshere.

Yes, Docker offersRegistry as a pull through cache,unfortunatelyit only covers the DockerHub case. It won't cache images fromquay.io,registry.k8s.io,gcr.io, or any such, including any private registries.

That means that your shiny new Kubernetes cluster is now a bandwidth hog, since every image will be pulled from theInternet on every Node it runs on, with no reuse.

This is due to the way the Docker "client" implements--registry-mirror, it only ever contacts mirrors for imageswith no repository reference (eg, from DockerHub).When a repository is specifieddockerd goes directly there, via HTTPS (and also via HTTP if included in a--insecure-registry list), thus completely ignoring the configured mirror.

Yeah. Docker Inc should do it. So should NPM, Inc. Wonder why they don't. 😼

• Basic Docker-for-Mac set-up instructions
• Basic Docker-for-Windows set-up instructions.
• Test and make auth work with quay.io, unfortunately I don't have access to it (hint, hint, quay)
• Hide the mitmproxy building code under a Docker build ARG.
• "Developer Office" proxy scenario, where many developers on a fast LAN share a proxy for bandwidth and speed savings (already works for pulls, but messes up pushes, which developers tend to use a lot)