Bitbucket Pipelines allows you to build a Docker image from a Dockerfile in your repository and to push that to a Docker registry, by running Docker commands within your build pipeline. Dive straight in – the pipeline environment is provided by default and you don't need to customize it!

Enable access to Docker

To enable access to Docker daemon, you can either add docker as a service on the step (recommended), or add the global option in yourbitbucket-pipelines.yml.

Add Docker as a service in your build step (recommended)

Note that Docker does not need to be declared as a service in the definitions section. It is a default service that is provided by Pipelines without a definition.

Add Docker to all build steps in your repository

Note that even if you declare Docker here, it still counts as a service for Pipelines, has a limit of 1 GB memory, and can only be run with two other services in your build step. This setting is provided for legacy support, and we recommend setting it on a step level so there's no confusion about how many services you can run in your pipeline.

How it works

Configuring Docker as a service will:

• mount the Docker CLI executable in your build container

• run and provide your build access to a Docker daemon

You can verify this by runningdocker version:

You can check your bitbucket-pipelines.yml file with our online validator.

Docker BuildKit

BuildKit is enabled by default in Pipelines builds.

Disable Docker BuildKit

To disable Docker BuildKit in a Bitbucket Pipeline, set theDOCKER_BUILDKIT=0 environment variable in the pipeline configuration (bitbucket-pipelines.yml).

For example

For information on Docker BuildKit, visit:Docker Docs — Build images with BuildKit.

Docker BuildKit restrictions

To protect the security of our users, the following Docker BuildKit features have been disabled in addition to the features listed inRunning Docker commands:

• multi-architecture builds

• the--platform option (such asdocker run --platform linux/arm/v7)

• theDocker BuildKit Buildx plugin

The following DockerfileRUN directive options, also known as Dockerfile frontend syntaxes, have been disabled:

• RUN --mount=type=ssh — To access yourBitbucket Pipelines SSH keys, use the--ssh option with theBITBUCKET_SSH_KEY_FILEvariable, such as--ssh default=$BITBUCKET_SSH_KEY_FILE

• RUN --security=insecure

For information on BuildKit Dockerfile frontend syntaxes, visit:Docker BuildKit GitHub repository — Dockerfile frontend syntaxes.

Docker BuildKit caching limitations

Thepredefined docker cache used for caching the layers produced during Docker Build operations does not cache layers produced when using BuildKit.

TheRUN --mount=type=cacheDocker frontend syntax will only retain the cache until the pipeline step is complete; it will not be available for other steps in the pipeline or new pipeline runs.

Using secrets and secure variables with Docker BuildKit

Docker BuildKit includes secret handling; helping to keep your passwords, API keys, and other sensitive information out of the Docker images you generate. To use BuildKit secrets, use the--secretDocker Build option, and the--mount=type=secretBuildKit frontend syntax.

The following examples show how to use BuildKit secrets with:

• Bitbucket Pipelines Secure variables

• External secret managers or vaults

Use Bitbucket Pipelines Secure variables with BuildKit

Bitbucket PipelinesSecure variables can be passed directly to a BuildKit build using the--secret option, then the secret can be used inside the BuildKit build using the--mount=type=secret BuildKit frontend syntax.

For example, using a Bitbucket Pipelines Secure variable namedMY_SECRET, the pipeline step would be:

The secure variable then can be mounted on theRUN instruction from the Docker default secret store (/run/secrets/*) such as:

In this example, the secret will return a blank line in the pipeline log and would be printed in the container used to generate the image layer by thecat command.

Use externally sourced secrets with BuildKit in Bitbucket Pipelines

Secrets from external secret managers (such asHashiCorp Vault,Azure Key Vault, andGoogle Cloud Secret Manager) need to be stored into a file on the pipeline before they can be used. The file will be deleted once the pipeline step in complete and the container is removed.

For example, to useMY_OTHER _SECRET from an external provider; get the secret from the external provider, store it in a file, and pass it to the build using the--secret option. This example usesecho 'My secret API Key' instead of retrieving a secret from an external provider.

The secure variable can then be mounted on theRUN instruction from the custom secret location (/my_secret_file), such as:

In this example, the secret will return a blank line in the pipeline log and would be printed in the container used to generate the image layer by thecat command.

Troubleshooting Docker BuildKit

If you are experiencing issues Docker build issues due to Docker BuildKit, you can disable BuildKit by settingDOCKER_BUILDKIT=0 before thedocker command.

Such as:

Running Docker commands

Inside your Pipelines script you can run most Docker commands. See theDocker command line reference for information on how to use these commands.

These restrictions, including the restricted commands listed below, only apply to the pipelines executed on our cloud infrastructure. These restrictions don't apply to the self-hosted pipelineRunners.

We've had to restrict a few for security reasons, including:

• Docker Swarm-related commands

• mapping volumes with a source outside$BITBUCKET_CLONE_DIR

• the--platform option

• docker run --privileged

• docker run --mount

Full list of restricted commands

The security of your data is really important to us, especially when you are trusting it to the cloud. To keep everybody safe we've restricted the following:

Fordocker container run/docker run we don't allow:

• --cap-add

• --device

• --ipc

• --mount

• --pid

• --privileged

• --security-opt

• --userns

• --uts

• --volume,-v (other than/opt/atlassian/bitbucketci/agent/build/.* or/opt/atlassian/pipelines/agent/build/.*)

Fordocker container update/docker update we don't allow:

• --devices

Fordocker container exec/docker exec we don't allow:

• --privileged

Fordocker image build /docker build we don't allow:

• --security-opt

Using Docker Compose

If you'd like to use Docker Compose in your container, you''ll need to install a binary that is compatible with your specified build container.

Using an external Docker daemon

If you have configured your build to run commands against your own Docker daemon hosted elsewhere, you can continue to do so. In this case, you should provide your own CLI executable as part ofyour build image (rather than enabling Docker in Pipelines), so the CLI version is compatible with the daemon version you are running.

Docker layer caching

If you have added Docker as a service, you can also add a Docker cache to your steps. Adding the cache can speed up your build by reusing previously built layers and only creating new dynamic layers as required in the step.

A common use case for Docker cache is when you are building images. However, if you find that performance slows with the cache enabled, check you are notinvalidating the layers in your dockerfile.

Docker layer caches have the same limitations and behaviors as regular caches as described on Caching Dependencies.

Docker memory limits

By default, the Docker daemon in Pipelines has a total memory limit of 1024 MB. This allocation includes all containers run viadocker run commands, as well as the memory needed to executedocker build commands.

To increase the memory available to Docker you canchange the memory limit for the built-in docker service. Thememory parameter is a whole number of megabytes greater than 128 and not larger than the available memory for the step.

Below is a working example of how you can set memory limits to multiple Docker services and use the appropriate service depending on the step requirements.

In the example below, we are are giving the docker service twice the default allocation of 1024 MB (2048). Depending on your other services and whether you have configuredlarge builds for extra memory, you can increase this even further (learn more about memory limits).

Allocate available memory or set memory limits

If your pipeline uses an Atlassian Docker service to start a container, you can configure the amount of memory available to that container in your YAML file. If you don’t configure the memory available, the default memory limit for that Docker service will apply. For more details on memory limits in your Docker service, refer toRun Docker commands in Bitbucket Pipelines.

Many third-party tools you may be using, such as Java Virtual Machine (JVM), set their own limits on memory usage. Because of these limitations, you may need to set resource constraints on your container’s memory. For more information on resource constraints in Docker, check out theDocker doc on this exact topic.

The following examples provide what you need for setting memory limits for your Docker service which will help ensure your builds run successfully.

Example of setting memory limit via Docker CLI:

Example of setting memory limit via Docker Compose:

Authenticate when pushing to a registry

To push images to a registry, you need to use docker login to authenticate prior to callingdocker push. You should set your username and password usingvariables.

For example, add this to your pipeline script:

Reserved ports

Port restrictions

There are some reserved ports which can't be used:

• 29418