Repository files navigation

The code remains available for historic purposes.

The README as of the archival date remains unchanged below for historic purposes.

kaniko is a tool to build container images from a Dockerfile, inside a containeror Kubernetes cluster.

kaniko doesn't depend on a Docker daemon and executes each command within aDockerfile completely in userspace. This enables building container images inenvironments that can't easily or securely run a Docker daemon, such as astandard Kubernetes cluster.

kaniko is meant to be run as an image:gcr.io/kaniko-project/executor. We donot recommend running the kaniko executor binary in another image, as itmight not work as you expect - seeKnown Issues.

We'd love to hear from you! Join us on#kaniko Kubernetes Slack

📣Please fill out ourquick 5-question survey so that we canlearn how satisfied you are with kaniko, and what improvements we should make.Thank you! 👯

If you are interested in contributing to kaniko, seeDEVELOPMENT.md andCONTRIBUTING.md.

Table of Contentsgenerated withDocToc

• kaniko - Build Images In Kubernetes
  • 🚨NOTE: kaniko is not an officially supported Google product🚨
  • Community
  • How does kaniko work?
  • Known Issues
  • Demo
  • Tutorial
  • Using kaniko
    • kaniko Build Contexts
    • Using Azure Blob Storage
    • Using Private Git Repository
    • Using Standard Input
    • Running kaniko
      • Running kaniko in a Kubernetes cluster
        • Kubernetes secret
      • Running kaniko in gVisor
      • Running kaniko in Google Cloud Build
      • Running kaniko in Docker
    • Caching
      • Caching Layers
      • Caching Base Images
    • Pushing to Different Registries
      • Pushing to Docker Hub
      • Pushing to Google GCR
      • Pushing to GCR using Workload Identity
      • Pushing to Amazon ECR
      • Pushing to Azure Container Registry
      • Pushing to JFrog Container Registry or to JFrog Artifactory
    • Additional Flags
      • Flag--build-arg
      • Flag--cache
      • Flag--cache-dir
      • Flag--cache-repo
      • Flag--cache-copy-layers
      • Flag--cache-run-layers
      • Flag--cache-ttl duration
      • Flag--cleanup
      • Flag--compressed-caching
      • Flag--context-sub-path
      • Flag--custom-platform
      • Flag--digest-file
      • Flag--dockerfile
      • Flag--force
      • Flag--git
      • Flag--image-name-with-digest-file
      • Flag--image-name-tag-with-digest-file
      • Flag--insecure
      • Flag--insecure-pull
      • Flag--insecure-registry
      • Flag--label
      • Flag--log-format
      • Flag--log-timestamp
      • Flag--no-push
      • Flag--no-push-cache
      • Flag--oci-layout-path
      • Flag--push-retry
      • Flag--registry-certificate
      • Flag--registry-client-cert
      • Flag--registry-map
      • Flag--registry-mirror
      • Flag--skip-default-registry-fallback
      • Flag--reproducible
      • Flag--single-snapshot
      • Flag--skip-push-permission-check
      • Flag--skip-tls-verify
      • Flag--skip-tls-verify-pull
      • Flag--skip-tls-verify-registry
      • Flag--skip-unused-stages
      • Flag--snapshot-mode
      • Flag--tar-path
      • Flag--target
      • Flag--use-new-run
      • Flag--verbosity
      • Flag--ignore-var-run
      • Flag--ignore-path
      • Flag--image-fs-extract-retry
      • Flag--image-download-retry
    • Debug Image
  • Security
    • Verifying Signed Kaniko Images
  • Kaniko Builds - Profiling
  • Creating Multi-arch Container Manifests Using Kaniko and Manifest-tool
    • General Workflow
    • Limitations and Pitfalls
    • Example CI Pipeline (GitLab)
      • Building the Separate Container Images
      • Merging the Container Manifests
      • On the Note of Adding Versioned Tags
  • Comparison with Other Tools
  • Community
  • Limitations
    • mtime and snapshotting
  • References

We'd love to hear from you! Join#kaniko on Kubernetes Slack

The kaniko executor image is responsible for building an image from a Dockerfileand pushing it to a registry. Within the executor image, we extract thefilesystem of the base image (the FROM image in the Dockerfile). We then executethe commands in the Dockerfile, snapshotting the filesystem in userspace aftereach one. After each command, we append a layer of changed files to the baseimage (if there are any) and update image metadata.

• kaniko does not support building Windows containers.
• Running kaniko in any Docker image other than the official kaniko image is notsupported due to implementation details.
  • This includes copying the kaniko executables from the official image intoanother image (e.g. a Jenkins CI agent).
  • In particular, it cannot use chroot or bind-mount because its container mustnot require privilege, so it unpacks directly into its own container rootand may overwrite anything already there.
• kaniko does not support the v1 Registry API(Registry v1 API Deprecation)

For a detailed example of kaniko with local storage, please refer to agetting started tutorial.

Please seeReferences for more docs & video tutorials

To use kaniko to build and push an image for you, you will need:

1. Abuild context, aka something to build
2. Arunning instance of kaniko

kaniko's build context is very similar to the build context you would send yourDocker daemon for an image build; it represents a directory containing aDockerfile which kaniko will use to build your image. For example, aCOPYcommand in your Dockerfile should refer to a file in the build context.

You will need to store your build context in a place that kaniko can access.Right now, kaniko supports these storage solutions:

• GCS Bucket
• S3 Bucket
• Azure Blob Storage
• Local Directory
• Local Tar
• Standard Input
• Git Repository

Note about Local Directory: this option refers to a directory within the kanikocontainer. If you wish to use this option, you will need to mount in your buildcontext into the container as a directory.

Note about Local Tar: this option refers to a tar gz file within the kanikocontainer. If you wish to use this option, you will need to mount in your buildcontext into the container as a file.

Note about Standard Input: the only Standard Input allowed by kaniko is in.tar.gz format.

If using a GCS or S3 bucket, you will first need to create a compressed tar ofyour build context and upload it to your bucket. Once running, kaniko will thendownload and unpack the compressed tar of the build context before starting theimage build.

To create a compressed tar, you can run:

tar -C<path to build context> -zcvf context.tar.gz.

Then, copy over the compressed tar into your bucket. For example, we can copyover the compressed tar to a GCS bucket with gsutil:

gsutil cp context.tar.gz gs://<bucket name>

When running kaniko, use the--context flag with the appropriate prefix tospecify the location of your build context:

If you don't specify a prefix, kaniko will assume a local directory. Forexample, to use a GCS bucket calledkaniko-bucket, you would pass in--context=gs://kaniko-bucket/path/to/context.tar.gz.

If you are using Azure Blob Storage for context file, you will need to passAzure Storage Account Access Keyas an environment variable namedAZURE_STORAGE_ACCESS_KEY through KubernetesSecrets

You can usePersonal Access Tokens for Build Contexts from PrivateRepositories fromGitHub.

You can either pass this in as part of the git URL (e.g.,git://TOKEN@github.com/acme/myproject.git#refs/heads/mybranch) or using theenvironment variableGIT_TOKEN.

You can also passGIT_USERNAME andGIT_PASSWORD (password being the token)if you want to be explicit about the username.

If running kaniko and using Standard Input build context, you will need to addthe docker or kubernetes-i, --interactive flag. Once running, kaniko willthen get the data fromSTDIN and create the build context as a compressed tar.It will then unpack the compressed tar of the build context before starting theimage build. If no data is piped during the interactive run, you will need tosend the EOF signal by yourself by pressingCtrl+D.

Complete example of how to interactively run kaniko with.tar.gz StandardInput data, using docker:

echo -e'FROM alpine \nRUN echo "created from standard input"'> Dockerfile| tar -cf - Dockerfile| gzip -9| docker run \  --interactive -v$(pwd):/workspace gcr.io/kaniko-project/executor:latest \  --context tar://stdin \  --destination=<gcr.io/$project/$image:$tag>

Complete example of how to interactively run kaniko with.tar.gz StandardInput data, using Kubernetes command line with a temporary container andcompletely dockerless:

echo -e'FROM alpine \nRUN echo "created from standard input"'> Dockerfile| tar -cf - Dockerfile| gzip -9| kubectl run kaniko \--rm --stdin=true \--image=gcr.io/kaniko-project/executor:latest --restart=Never \--overrides='{  "apiVersion": "v1",  "spec": {    "containers": [      {        "name": "kaniko",        "image": "gcr.io/kaniko-project/executor:latest",        "stdin": true,        "stdinOnce": true,        "args": [          "--dockerfile=Dockerfile",          "--context=tar://stdin",          "--destination=gcr.io/my-repo/my-image"        ],        "volumeMounts": [          {            "name": "cabundle",            "mountPath": "/kaniko/ssl/certs/"          },          {            "name": "docker-config",            "mountPath": "/kaniko/.docker/"          }        ]      }    ],    "volumes": [      {        "name": "cabundle",        "configMap": {          "name": "cabundle"        }      },      {        "name": "docker-config",        "configMap": {          "name": "docker-config"        }      }    ]  }}'

There are several different ways to deploy and run kaniko:

• In a Kubernetes cluster
• In gVisor
• In Google Cloud Build
• In Docker

Requirements:

• Standard Kubernetes cluster (e.g. usingGKE)
• Kubernetes Secret
• Abuild context

To run kaniko in a Kubernetes cluster, you will need a standard runningKubernetes cluster and a Kubernetes secret, which contains the auth required topush the final image.

To create a secret to authenticate to Google Cloud Registry, follow these steps:

1. Create a service account in the Google Cloud Console project you want to pushthe final image to withStorage Admin permissions.
2. Download a JSON key for this service account
3. Rename the key tokaniko-secret.json
4. To create the secret, run:

kubectl create secret generic kaniko-secret --from-file=<path to kaniko-secret.json>

Note: If using a GCS bucket in the same GCP project as a build context, thisservice account should now also have permissions to read from that bucket.

The Kubernetes Pod spec should look similar to this, with the args parametersfilled in:

apiVersion:v1kind:Podmetadata:name:kanikospec:containers:    -name:kanikoimage:gcr.io/kaniko-project/executor:latestargs:        -"--dockerfile=<path to Dockerfile within the build context>"        -"--context=gs://<GCS bucket>/<path to .tar.gz>"        -"--destination=<gcr.io/$PROJECT/$IMAGE:$TAG>"volumeMounts:        -name:kaniko-secretmountPath:/secretenv:        -name:GOOGLE_APPLICATION_CREDENTIALSvalue:/secret/kaniko-secret.jsonrestartPolicy:Nevervolumes:    -name:kaniko-secretsecret:secretName:kaniko-secret

This example pulls the build context from a GCS bucket. To use a local directorybuild context, you could consider using configMaps to mount in small buildcontexts.

Running kaniko ingVisor provides anadditional security boundary. You will need to add the--force flag to runkaniko in gVisor, since currently there isn't a way to determine whether or nota container is running in gVisor.

docker run --runtime=runsc -v$(pwd):/workspace -v~/.config:/root/.config \gcr.io/kaniko-project/executor:latest \--dockerfile=<path to Dockerfile> --context=/workspace \--destination=gcr.io/my-repo/my-image --force

We pass in--runtime=runsc to use gVisor. This example mounts the currentdirectory to/workspace for the build context and the~/.config directoryfor GCR credentials.

Requirements:

• Abuild context

To run kaniko in GCB, add it to your build config as a build step:

steps:  -name:gcr.io/kaniko-project/executor:latestargs:["--dockerfile=<path to Dockerfile within the build context>","--context=dir://<path to build context>","--destination=<gcr.io/$PROJECT/$IMAGE:$TAG>",]

kaniko will build and push the final image in this build step.

Requirements:

• Docker

We can run the kaniko executor image locally in a Docker daemon to build andpush an image from a Dockerfile.

For example, when using gcloud and GCR you could run kaniko as follows:

docker run \    -v"$HOME"/.config/gcloud:/root/.config/gcloud \    -v /path/to/context:/workspace \    gcr.io/kaniko-project/executor:latest \    --dockerfile /workspace/Dockerfile \    --destination"gcr.io/$PROJECT_ID/$IMAGE_NAME:$TAG" \    --context dir:///workspace/

There is also a utility scriptrun_in_docker.sh that canbe used as follows:

./run_in_docker.sh<path to Dockerfile><path to build context><destination of final image>

NOTE:run_in_docker.sh expects a path to a Dockerfile relative to theabsolute path of the build context.

An example run, specifying the Dockerfile in the container directory/workspace, the build context in the local directory/home/user/kaniko-project, and a Google Container Registry as a remote imagedestination:

./run_in_docker.sh /workspace/Dockerfile /home/user/kaniko-project gcr.io/$PROJECT_ID/$TAG

kaniko can cache layers created byRUN(configured by flag--cache-run-layers) andCOPY (configured by flag--cache-copy-layers)commands in a remote repository. Before executing a command, kaniko checks thecache for the layer. If it exists, kaniko will pull and extract the cached layerinstead of executing the command. If not, kaniko will execute the command andthen push the newly created layer to the cache.

Note that kaniko cannot read layers from the cache after a cache miss: once alayer has not been found in the cache, all subsequent layers are built locallywithout consulting the cache.

Users can opt into caching by setting the--cache=true flag. A remoterepository for storing cached layers can be provided via the--cache-repoflag. If this flag isn't provided, a cached repo will be inferred from the--destination provided.

kaniko can cache images in a local directory that can be volume mounted into thekaniko pod. To do so, the cache must first be populated, as it is read-only. Weprovide a kaniko cache warming image atgcr.io/kaniko-project/warmer:

docker run -v$(pwd):/workspace gcr.io/kaniko-project/warmer:latest --cache-dir=/workspace/cache --image=<image to cache> --image=<another image to cache>docker run -v$(pwd):/workspace gcr.io/kaniko-project/warmer:latest --cache-dir=/workspace/cache --dockerfile=<path to dockerfile>docker run -v$(pwd):/workspace gcr.io/kaniko-project/warmer:latest --cache-dir=/workspace/cache --dockerfile=<path to dockerfile> --build-arg version=1.19

--image can be specified for any number of desired images.--dockerfile canbe specified for the path of dockerfile for cache.These command will combined tocache those images by digest in a local directory namedcache. Once the cacheis populated, caching is opted into with the same--cache=true flag as above.The location of the local cache is provided via the--cache-dir flag,defaulting to/cache as with the cache warmer. See theexamples directoryfor how to use with kubernetes clusters and persistent cache volumes.

kaniko uses Docker credential helpers to push images to a registry.

kaniko comes with support for GCR, Dockerconfig.json and Amazon ECR, butconfiguring another credential helper should allow pushing to a differentregistry.

Get your docker registry user and password encoded in base64

echo -n USER:PASSWORD | base64

Create aconfig.json file with your Docker registry url and the previousgenerated base64 string

Note: Please use v1 endpoint. See #1209 for more details

{"auths": {"https://index.docker.io/v1/": {"auth":"xxxxxxxxxxxxxxx"    }  }}

Run kaniko with theconfig.json inside/kaniko/.docker/config.json

docker run -ti --rm -v`pwd`:/workspace -v`pwd`/config.json:/kaniko/.docker/config.json:ro gcr.io/kaniko-project/executor:latest --dockerfile=Dockerfile --destination=yourimagename

To create a credentials to authenticate to Google Cloud Registry, follow thesesteps:

1. Create aservice accountor in the Google Cloud Console project you want to push the final image towithStorage Admin permissions.
2. Download a JSON key for this service account
3. (optional) Rename the key tokaniko-secret.json, if you don't rename, youhave to change the name used the command(in the volume part)
4. Run the container adding the path in GOOGLE_APPLICATION_CREDENTIALS env var

docker run -ti --rm -e GOOGLE_APPLICATION_CREDENTIALS=/kaniko/config.json \-v`pwd`:/workspace -v`pwd`/kaniko-secret.json:/kaniko/config.json:ro gcr.io/kaniko-project/executor:latest \--dockerfile=Dockerfile --destination=yourimagename

If you have enabled Workload Identity on your GKE cluster then you can use theworkload identity to push built images to GCR without adding aGOOGLE_APPLICATION_CREDENTIALS in your kaniko pod specification.

Learn more on how toenableandmigrate existing appsto workload identity.

To authenticate using workload identity you need to run the kaniko pod using theKubernetes Service Account (KSA) bound to Google Service Account (GSA) which hasStorage.Admin permissions to push images to Google Container registry.

Please follow the detailed stepshereto create a Kubernetes Service Account, Google Service Account and create an IAMpolicy binding between the two to allow the Kubernetes Service account to act asthe Google service account.

To grant the Google Service account the right permission to push to GCR, run thefollowing GCR command

gcloud projects add-iam-policy-binding $PROJECT \  --member=serviceAccount:[gsa-name]@${PROJECT}.iam.gserviceaccount.com \  --role=roles/storage.objectAdmin

Please ensure, kaniko pod is running in the namespace and with a KubernetesService Account.

The Amazon ECRcredential helper isbuilt into the kaniko executor image.

1. Configure credentials

   1. You can use instance roles when pushing to ECR from a EC2 instance or fromEKS, byconfiguring the instance role permissions(the AWS managed policyEC2InstanceProfileForImageBuilderECRContainerBuilds provides broadpermissions to upload ECR images and may be used as configurationbaseline). Additionally, setAWS_SDK_LOAD_CONFIG=true as environmentvariable within the kaniko pod. If running on an EC2 instance with aninstance profile, you may also need to setAWS_EC2_METADATA_DISABLED=true for kaniko to pick up the correctcredentials.

   2. Or you can create a Kubernetes secret for your~/.aws/credentials fileso that credentials can be accessed within the cluster. To create thesecret, run:shell kubectl create secret generic aws-secret --from-file=<path to .aws/credentials>

The Kubernetes Pod spec should look similar to this, with the args parametersfilled in. Note thataws-secret volume mount and volume are only needed whenusing AWS credentials from a secret, not when using instance roles.

apiVersion:v1kind:Podmetadata:name:kanikospec:containers:    -name:kanikoimage:gcr.io/kaniko-project/executor:latestargs:        -"--dockerfile=<path to Dockerfile within the build context>"        -"--context=s3://<bucket name>/<path to .tar.gz>"        -"--destination=<aws_account_id.dkr.ecr.region.amazonaws.com/my-repository:my-tag>"volumeMounts:# when not using instance role        -name:aws-secretmountPath:/root/.aws/restartPolicy:Nevervolumes:# when not using instance role    -name:aws-secretsecret:secretName:aws-secret

An ACRcredential helperis built into the kaniko executor image, which can be used to authenticate withwell-known Azure environmental information.

To configure credentials, you will need to do the following:

1. Update thecredStore section ofconfig.json:

{"credsStore":"acr" }

A downside of this approach is that ACR authentication will be used for allregistries, which will fail if you also pull from DockerHub, GCR, etc. Thus, itis better to configure the credential tool only for your ACR registries by usingcredHelpers instead ofcredsStore:

{"credHelpers": {"mycr.azurecr.io":"acr-env" } }

You can mount in the new config as a configMap:

kubectl create configmap docker-config --from-file=<path to config.json>

2. Configure credentials

You can create a Kubernetes secret with environment variables required forService Principal authentication and expose them to the builder container.

AZURE_CLIENT_ID=<clientID>AZURE_CLIENT_SECRET=<clientSecret>AZURE_TENANT_ID=<tenantId>

If the above are not set then authentication falls back to managed serviceidentities and the MSI endpoint is attempted to be contacted which will work invarious Azure contexts such as App Service and Azure Kubernetes Service wherethe MSI endpoint will authenticate the MSI context the service is running under.

The Kubernetes Pod spec should look similar to this, with the args parametersfilled in. Note thatazure-secret secret is only needed when using AzureService Principal credentials, not when using a managed service identity.

apiVersion:v1kind:Podmetadata:name:kanikospec:containers:    -name:kanikoimage:gcr.io/kaniko-project/executor:latestargs:        -"--dockerfile=<path to Dockerfile within the build context>"        -"--context=s3://<bucket name>/<path to .tar.gz>"        -"--destination=mycr.azurecr.io/my-repository:my-tag"envFrom:# when authenticating with service principal        -secretRef:name:azure-secretvolumeMounts:        -name:docker-configmountPath:/kaniko/.docker/volumes:    -name:docker-configconfigMap:name:docker-configrestartPolicy:Never

Kaniko can be used with bothJFrog Container Registryand JFrog Artifactory.

Get your JFrog Artifactory registry user and password encoded in base64

echo -n USER:PASSWORD | base64

Create aconfig.json file with your Artifactory Docker local registry URL andthe previous generated base64 string

{"auths": {"artprod.company.com": {"auth":"xxxxxxxxxxxxxxx"    }  }}

For example, for Artifactory cloud users, the docker registry should be:<company>.<local-repository-name>.io.

Run kaniko with theconfig.json inside/kaniko/.docker/config.json

docker run -ti --rm -v `pwd`:/workspace -v `pwd`/config.json:/kaniko/.docker/config.json:ro gcr.io/kaniko-project/executor:latest --dockerfile=Dockerfile --destination=yourimagename

After the image is uploaded, using the JFrog CLI, you cancollectandpublishthe build information to Artifactory and triggerbuild vulnerabilities scanningusing JFrog Xray.

To collect and publish the image's build information using the JenkinsArtifactory plugin, see instructions forscripted pipelineanddeclarative pipeline.

This flag allows you to pass in ARG values at build time, similarly to Docker.You can set it multiple times for multiple arguments.

Note that passing values that contain spaces is not natively supported - youneed to ensure that the IFS is set to null before your executor command. You canset this by addingexport IFS='' before your executor call. See the followingexample

export IFS=''/kaniko/executor --build-arg"MY_VAR='value with spaces'" ...

Set this flag as--cache=true to opt into caching with kaniko.

Set this flag to specify a local directory cache for base images. Defaults to/cache.

This flag must be used in conjunction with the--cache=true flag.

Set this flag to specify a remote repository that will be used to store cachedlayers.

If this flag is not provided, a cache repo will be inferred from the--destination flag. If--destination=gcr.io/kaniko-project/test, then cachedlayers will be stored ingcr.io/kaniko-project/test/cache.

This flag must be used in conjunction with the--cache=true flag.

Set this flag to cache copy layers.

Set this flag to cache run layers (default=true).

Cache timeout in hours. Defaults to two weeks.

Set this flag to clean the filesystem at the end of the build.

Set this to false in order to prevent tar compression for cached layers. Thiswill increase the runtime of the build, but decrease the memory usage especiallyfor large builds. Try to use--compressed-caching=false if your build failswith an out of memory error. Defaults to true.

Set a sub path within the given--context.

Its particularly useful when your context is, for example, a git repository, andyou want to build one of its subfolders instead of the root folder.

Allows to build with another default platform than the host, similarly to dockerbuild --platform xxx the value has to be on the form--custom-platform=linux/arm, with acceptable values listed here:GOOS/GOARCH.

It's also possible specifying CPU variants adding it as a third parameter (like--custom-platform=linux/arm/v5). Currently CPU variants are only known to beused for the ARM architecture as listed here:GOARM

The resulting images cannot provide any metadata about CPU variant due to alimitation of the OCI-image specification.

This is not virtualization and cannot help to build an architecture notnatively supported by the build host. This is used to build i386 on an amd64Host for example, or arm32 on an arm64 host.

Set this flag to specify a file in the container. This file will receive thedigest of a built image. This can be used to automatically track the exact imagebuilt by kaniko.

For example, setting the flag to--digest-file=/dev/termination-log will writethe digest to that file, which is picked up by Kubernetes automatically as the{{.state.terminated.message}} of the container.

Path to the dockerfile to be built. (default "Dockerfile")

Force building outside of a container

Branch to clone if build context is a git repository (defaultbranch=,single-branch=false,recurse-submodules=false,insecure-skip-tls=false)

Specify a file to save the image name w/ digest of the built image to.

Specify a file to save the image name w/ image tag and digest of the built imageto.

Set this flag if you want to push images to a plain HTTP registry. It issupposed to be used for testing purposes only and should not be used inproduction!

Set this flag if you want to pull images from a plain HTTP registry. It issupposed to be used for testing purposes only and should not be used inproduction!

You can set--insecure-registry <registry-name> to use plain HTTP requestswhen accessing the specified registry. It is supposed to be used for testingpurposes only and should not be used in production! You can set it multipletimes for multiple registries.

Set this flag as--label key=value to set some metadata to the final image.This is equivalent as using theLABEL within the Dockerfile.

Set this flag as--log-format=<text|color|json> to set the log format.Defaults tocolor.

Set this flag as--log-timestamp=<true|false> to add timestamps to<text|color> log format. Defaults tofalse.

Set this flag if you only want to build the image, without pushing to aregistry. This can also be defined throughKANIKO_NO_PUSH environmentvariable.

NOTE: this will still push cache layers to the repo, to disable pushing cache layers use--no-push-cache

Set this flag if you do not want to push cache layers to aregistry. Can be used in addition to--no-push to push no layers to a registry.

Set this flag to specify a directory in the container where the OCI image layoutof a built image will be placed. This can be used to automatically track theexact image built by kaniko.

For example, to surface the image digest built in aTekton task,this flag should be set to match the image resourceoutputImageDir.

Note: Depending on the built image, the media type of the image manifest mightbe eitherapplication/vnd.oci.image.manifest.v1+json orapplication/vnd.docker.distribution.manifest.v2+json.

Set this boolean flag totrue if you want the Kaniko process to exit withsuccess when a push error related to tag immutability occurs.

This is useful for example if you have parallel builds pushing the same tagand do not care which one actually succeeds.

Defaults tofalse.

Set this flag to the number of retries that should happen for the push of animage to a remote destination. Defaults to0.

Set this flag to provide a certificate for TLS communication with a givenregistry.

Expected format ismy.registry.url=/path/to/the/certificate.cert

Set this flag to provide a certificate/key pair for mutual TLS (mTLS)communication with a givenregistry that requires mTLSfor authentication.

Expected format ismy.registry.url=/path/to/client/cert.crt,/path/to/client/key.key

Set this flag if you want to remap registries references. Usefull for air gapenvironement for example. You can use this flag more than once, if you want toset multiple mirrors for a given registry. You can mention several remap in asingle flag too, separated by semi-colon. If an image is not found on the firstmirror, Kaniko will try the next mirror(s), and at the end fallback on theoriginal registry.

Registry maps can also be defined throughKANIKO_REGISTRY_MAP environmentvariable.

Expected format isoriginal-registry=remapped-registry[;another-reg=another-remap[;...]] forexample.

Note that youcan specify a URL with scheme for this flag. Some valid optionsare:

• index.docker.io=mirror.gcr.io
• gcr.io=127.0.0.1
• quay.io=192.168.0.1:5000
• index.docker.io=docker-io.mirrors.corp.net;index.docker.io=mirror.gcr.io;gcr.io=127.0.0.1will trydocker-io.mirrors.corp.net thenmirror.gcr.io forindex.docker.io and127.0.0.1 forgcr.io
• docker.io=harbor.provate.io/theproject

Set this flag if you want to use a registry mirror instead of the defaultindex.docker.io. You can use this flag more than once, if you want to setmultiple mirrors. If an image is not found on the first mirror, Kaniko will trythe next mirror(s), and at the end fallback on the default registry.

Mirror can also be defined throughKANIKO_REGISTRY_MIRROR environmentvariable.

Expected format ismirror.gcr.io ormirror.gcr.io/path for example.

Note that youcan specify a URL with scheme for this flag. Some valid optionsare:

• mirror.gcr.io
• 127.0.0.1
• 192.168.0.1:5000
• mycompany-docker-virtual.jfrog.io
• harbor.provate.io/theproject

Set this flag if you want the build process to fail if none of the mirrorslisted in flagregistry-mirror can pull some image.This should be used with mirrors that implements a whitelist or some imagerestrictions.

Ifregistry-mirror is not set or is empty, this flagis ignored.

Set this flag to strip timestamps out of the built image and make itreproducible.

This flag takes a single snapshot of the filesystem at the end of the build, soonly one layer will be appended to the base image.

Set this flag to skip push permission check. This can be useful to delay Kanikosfirst request for delayed network-policies.

Set this flag to skip TLS certificate validation when pushing to a registry. Itis supposed to be used for testing purposes only and should not be used inproduction!

Set this flag to skip TLS certificate validation when pulling from a registry.It is supposed to be used for testing purposes only and should not be used inproduction!

You can set--skip-tls-verify-registry <registry-name> to skip TLS certificatevalidation when accessing the specified registry. It is supposed to be used fortesting purposes only and should not be used in production! You can set itmultiple times for multiple registries.

This flag builds only used stages if defined totrue. Otherwise it builds bydefault all stages, even the unnecessary ones until it reaches the target stage/ end of Dockerfile

You can set the--snapshot-mode=<full (default), redo, time> flag to set howkaniko will snapshot the filesystem.

• If--snapshot-mode=full is set, the full file contents and metadata areconsidered when snapshotting. This is the least performant option, but alsothe most robust.

• If--snapshot-mode=redo is set, the file mtime, size, mode, owner uid andgid will be considered when snapshotting. This may be up to 50% faster than"full", particularly if your project has a large number files.

• If--snapshot-mode=time is set, only file mtime will be considered whensnapshotting (seelimitations related to mtime).

Set this flag as--tar-path=<path> to save the image as a tarball at path. Youneed to set--destination as well (for example--destination=image). If youwant to save the image as tarball only you also need to set--no-push.

Set this flag to indicate which build stage is the target build stage.

Using this flag enables an experimental implementation of the Run command whichdoes not rely on snapshotting at all. In this approach, in order to computewhich files were changed, a marker file is created before executing the Runcommand. Then the entire filesystem is walked (takes ~1-3 seconds for 700Kfiles)to find all files whose ModTime is greater than the marker file. With this newrun command implementation, the total build time is reduced seeing performanceimprovements in the range of ~75%. This new run mode trades offaccuracy/correctness in some cases (potential for missed files in a "snapshot")for improved performance by avoiding the full filesystem snapshots.

Set this flag as--verbosity=<panic|fatal|error|warn|info|debug|trace> to setthe logging level. Defaults toinfo.

Ignore /var/run when taking image snapshot. Set it to false to preserve/var/run/* in destination image. (Default true).

Set this flag as--ignore-path=<path> to ignore path when taking an imagesnapshot. Set it multiple times for multiple ignore paths.

Set this flag to the number of retries that should happen for the extracting animage filesystem. Defaults to0.

Set this flag to the number of retries that should happen when downloading theremote image. Consecutive retries occur with exponential backoff and an initialdelay of 1 second. Defaults to 0`.

The kaniko executor image is based on scratch and doesn't contain a shell. Weprovidegcr.io/kaniko-project/executor:debug, a debug image which consists ofthe kaniko executor image along with a busybox shell to enter.

You can launch the debug image with a shell entrypoint:

docker run -it --entrypoint=/busybox/sh gcr.io/kaniko-project/executor:debug

kaniko by itselfdoes not make it safe to run untrusted builds inside yourcluster, or anywhere else.

kaniko relies on the security features of your container runtime to providebuild security.

The minimum permissions kaniko needs inside your container are governed by a fewthings:

• The permissions required to unpack your base image into its container
• The permissions required to execute the RUN commands inside the container

If you have a minimal base image (SCRATCH or similar) that doesn't requirepermissions to unpack, and your Dockerfile doesn't execute any commands as theroot user, you can run kaniko without root permissions. It should be noted thatDocker runs as root by default, so you still require (in a sense) privileges touse kaniko.

You may be able to achieve the same default seccomp profile that Docker uses inyour Pod by settingseccompprofiles with annotations on aPodSecurityPolicyto create or update security policies on your cluster.

kaniko images are signed for versions >= 1.5.2 usingcosign!

To verify a public image, installcosignand use the providedpublic key:

$ cat cosign.pub-----BEGIN PUBLIC KEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE9aAfAcgAxIFMTstJUv8l/AMqnSKwP+vLu3NnnBDHCfREQpV/AJuiZ1UtgGpFpHlJLCNPmFkzQTnfyN5idzNl6Q==-----END PUBLIC KEY-----$ cosign verify -key ./cosign.pub gcr.io/kaniko-project/executor:latest

If your builds are taking long, we recently added support to analyze kanikofunction calls usingSlow Jam To startprofiling,

1. Add an environment variableSTACKLOG_PATH to yourpod definition.
2. If you are using the kanikodebug image, you can copy the file in thepre-stop container lifecycle hook.

While Kaniko itself currently does not support creating multi-arch manifests(contributions welcome), one can use tools such asmanifest-tool to stitch multipleseparate builds together into a single container manifest.

The general workflow for creating multi-arch manifests is as follows:

1. Build separate container images using Kaniko on build hosts matching yourtarget architecture and tag them with the appropriate ARCH tag.
2. Push the separate images to your container registry.
3. Manifest-tool identifies the separate manifests in your container registry,according to a given template.
4. Manifest-tool pushes a combined manifest referencing the separate manifests.

The following conditions must be met:

1. You need access to build-machines running the desired architectures (runningKaniko in an emulator, e.g. QEMU should also be possible but goes beyond thescope of this documentation). This is something to keep in mind when usingSaaS build tools such as github.com or gitlab.com, of which at the time ofwriting neither supports any non-x86_64 SaaS runners(GitHub,GitLab),so be prepared to bring your own machines(GitHub,GitLab.
2. Kaniko needs to be able to run on the desired architectures. At the time ofwriting, the official Kaniko container supportslinux/amd64, linux/arm64, linux/s390x and linux/ppc64le (not on *-debug images).
3. The container registry of your choice must be OCIv1 or Docker v2.2compatible.

It is up to you to find an automation tool that suits your needs best. Werecommend using a modern CI/CD system such as GitHub workflows or GitLab CI. Aswe (the authors) happen to use GitLab CI, the following examples are tailored tothis specific platform but the underlying principles should apply anywhere elseand the examples are kept simple enough, so that you should be able to followalong, even without any previous experiences with this specific platform. Whenin doubt, visit thegitlab-ci.yml reference pagefor a comprehensive overview of the GitLab CI keywords.

gitlab-ci.yml:

# define a job for building the containersbuild-container:stage:container-build# run parallel builds for the desired architecturesparallel:matrix:      -ARCH:amd64      -ARCH:arm64tags:# run each build on a suitable, preconfigured runner (must match the target architecture)    -runner-${ARCH}image:name:gcr.io/kaniko-project/executor:debugentrypoint:[""]script:# build the container image for the current arch using kaniko    ->-      /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile      "${CI_PROJECT_DIR}/Dockerfile" # push the image to the GitLab container      registry, add the current arch as tag. --destination      "${CI_REGISTRY_IMAGE}:${ARCH}"

gitlab-ci.yml:

# define a job for creating and pushing a merged manifestmerge-manifests:stage:container-build# all containers must be build before merging them# alternatively the job may be configured to run in a later stageneeds:    -job:container-buildartifacts:falsetags:# may run on any architecture supported by manifest-tool image    -runner-xyzimage:name:mplatform/manifest-tool:alpineentrypoint:[""]script:    ->-      manifest-tool # authorize against your container registry      --username=${CI_REGISTRY_USER} --password=${CI_REGISTRY_PASSWORD} push      from-args # define the architectures you want to merge --platforms      linux/amd64,linux/arm64 # "ARCH" will be automatically replaced by      manifest-tool # with the appropriate arch from the platform definitions      --template ${CI_REGISTRY_IMAGE}:ARCH # The name of the final, combined      image which will be pushed to your registry --target ${CI_REGISTRY_IMAGE}

For simplicity's sake we deliberately refrained from using versioned taggedimages (all builds will be tagged as "latest") in the previous examples, as wefeel like this adds to much platform and workflow specific code.

Nethertheless, for anyone interested in how we handle (dynamic) versioning inGitLab, here is a short rundown:

• If you are only interested in building tagged releases, you can simply use theGitLab predefinedCI_COMMIT_TAG variable when running a tag pipeline.
• When you (like us) want to additionally build container images outside ofreleases, things get a bit messier. In our case, we added a additional jobwhich runs before the build and merge jobs (don't forget to extend theneedssection of the build and merge jobs accordingly), which will set the tag tolatest when running on the default branch, to the commit hash when run onother branches and to the release tag when run on a tag pipeline.

gitlab-ci.yml:

container-get-tag:stage:pre-container-build-stagetags:    -runner-xyzimage:busyboxscript:# All other branches are tagged with the currently built commit SHA hash    -|      # If pipeline runs on the default branch: Set tag to "latest"      if test "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH"; then        tag="latest"      # If pipeline is a tag pipeline, set tag to the git commit tag      elif test -n "$CI_COMMIT_TAG"; then        tag="$CI_COMMIT_TAG"      # Else set the tag to the git commit sha      else        tag="$CI_COMMIT_SHA"      fi    -echo "tag=$tag" > build.env# parse tag to the build and merge jobs.# See: https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-jobartifacts:reports:dotenv:build.env

Similar tools include:

• BuildKit
• img
• orca-build
• umoci
• buildah
• FTL
• Bazel rules_docker

All of these tools build container images with different approaches.

BuildKit (andimg) can perform as a non-root user from within a container butrequires seccomp and AppArmor to be disabled to create nested containers.kaniko does not actually create nested containers, so it does not requireseccomp and AppArmor to be disabled. BuildKit supports "cross-building"multi-arch containers by leveraging QEMU.

orca-build depends onrunc to build images from Dockerfiles, which can notrun inside a container (for similar reasons toimg above).kaniko doesn'tuserunc so it doesn't require the use of kernel namespacing techniques.However,orca-build does not require Docker or any privileged daemon (sobuilds can be done entirely without privilege).

umoci works without any privileges, and also has no restrictions on the rootfilesystem being extracted (though it requires additional handling if yourfilesystem is sufficiently complicated). However, it has noDockerfile-likebuild tooling (it's a slightly lower-level tool that can be used to build suchbuilders -- such asorca-build).

Buildah specializes in building OCI images. Buildah's commands replicate allof the commands that are found in a Dockerfile. This allows building images withand without Dockerfiles while not requiring any root privileges. Buildah’sultimate goal is to provide a lower-level coreutils interface to build images.The flexibility of building images without Dockerfiles allows for theintegration of other scripting languages into the build process. Buildah followsa simple fork-exec model and does not run as a daemon but it is based on acomprehensive API in golang, which can be vendored into other tools.

FTL andBazel aim to achieve the fastest possible creation of Docker imagesfor a subset of images. These can be thought of as a special-case "fast path"that can be used in conjunction with the support for general Dockerfiles kanikoprovides.

kaniko-users Googlegroup

To Contribute to kaniko, seeDEVELOPMENT.md andCONTRIBUTING.md.

When taking a snapshot, kaniko's hashing algorithms include (or in the case of--snapshot-mode=time, only use) a file'smtime todetermine if the file has changed. Unfortunately, there is a delay between whenchanges to a file are made and when themtime is updated. This means:

• With the time-only snapshot mode (--snapshot-mode=time), kaniko may misschanges introduced byRUN commands entirely.
• With the default snapshot mode (--snapshot-mode=full), whether or not kanikowill add a layer in the case where aRUN command modifies a filebut thecontents do not change is theoretically non-deterministic. Thisdoes notaffect the contents which will still be correct, but it does affect thenumber of layers.

Note that these issues are currently theoretical only. If you see this issueoccur, pleaseopen an issue.

Kaniko currently supportsCOPY --chown andADD --chown Dockerfile command. It does not supportRUN --chown.

• Kaniko - Building Container Images In Kubernetes Without Docker.