Simple Dockerfile

FROM golang:1.21.3-alpineWORKDIR /appCOPY . .RUNgo build-v-o /usr/local/bin/myappCMD ["myapp"]

1. Identified Issues

1.1. Image size:
Thegolang:1.21.3-alpine base image carries a lot of stuff that make the image unnecessarily big. Since Golang is a compiled language, having the compiler and its dependencies in the final image is redundant.
Taking a simple Hello World sample for a instance, the image size amounts to 327MB.

1.2. Security Concerns:
We cannot not entirely ensure what is up in thegolang:1.21.3-alpine image.

1.3. Inefficiency:
Despite Docker's layer caching, theCOPY . . instruction invalidates the cache for all the subsequent changes. Consequently, any modification will result in downloading all the dependencies and full recompilation. the simple Hello World sample image would approximately take 10 seconds to build.

```#9 [5/5] RUN go build -v -o /usr/local/bin/myapp#9 0.503 go: downloading gorm.io/gorm v1.25.5#9 0.505 go: downloading gorm.io/driver/sqlite v1.5.4#9 0.723 go: downloading github.com/mattn/go-sqlite3 v1.14.17#9 0.913 go: downloading github.com/jinzhu/now v1.1.5#9 0.913 go: downloading github.com/jinzhu/inflection v1.0.0#9 1.182 internal/goos#9 1.182 internal/itoa#9 1.182 internal/godebugs...#9 8.840 gotour/example#9 DONE 9.6s```

2. Optimizations

2.1. Use a multi-stage build to reduce the image size and improve security.

#----- Builder stage -----#FROMgolang:1.21.3-alpineasbuilderWORKDIR /app# download dependenciesRUN--mount=type=bind,source=go.mod,target=go.mod\--mount=type=bind,source=go.sum,target=go.sum\    go mod download-x# compile go binaryRUN--mount=type=bind,source=.,target=.\CGO_ENABLED=0\    go build-o /usr/local/bin/myapp ./#----- Runtime stage -----#FROMgcr.io/distroless/static-debian11asrunnerCOPY --from=builder /usr/local/bin/myapp /usr/local/bin/myappEXPOSE 8080CMD ["myapp"]#----- Dev stage -----#FROMrunnerasdev#----- Prod stage -----#FROMrunnerasprod

First, we split the Dockerfile into two main stages:builder andrunner. Thebuilder is responsible for compiling the Go binary and therunner is responsible for running the binary. Therunner uses thedistroless/static-debian11 image as its base image which is a minimal image that contains only the necessary dependencies to run a statically compiled binary.
The image size is now reduced to 13.64MB.

Second, we seperate the download of dependencies and the compilation of the binary into two separate steps. Most of the time, the dependencies do not change as often as the source code. Therefore, we can only invalidate the docker layer cache when the dependencies change. This will save us a lot of time when building the image.

...# download dependenciesRUN --mount=type=bind,source=go.mod,target=go.mod \    --mount=type=bind,source=go.sum,target=go.sum \    go mod download -x# compile go binaryRUN --mount=type=bind,source=.,target=.\    CGO_ENABLED=0 \    go build -o /usr/local/bin/myapp ./...

2.2. Use docker dedicated RUN cache

However, a drawback arises when changing dependencies (go.mod), as Docker tends to redownload all dependencies and recompile the binary, leading to inefficiency.
In prior article:

[Go Tour] 1. Exploring the `go build` Command

Thao Nguyen ・ Dec 12 '23

#go#explore

I introduced how go buildkit caches the downloaded dependencies and compiled binaries. To capitalize on this feature, we can utilize Docker's dedicated RUN cache. The Dockerfile is as follows:

# download dependencies-RUN --mount=type=bind,source=go.mod,target=go.mod \+RUN \+   --mount=type=cache,target=/go/pkg/mod \+   --mount=type=bind,source=go.mod,target=go.mod \    --mount=type=bind,source=go.sum,target=go.sum \    go mod download -x# compile go binary-RUN --mount=type=bind,source=.,target=.\+RUN \+   --mount=type=cache,target=/go/pkg/mod \+   --mount=type=cache,target=/root/.cache/go-build \+   --mount=type=bind,source=.,target=.\    CGO_ENABLED=0 \    go build -v -o /usr/local/bin/myapp ./

By adding--mount=type=cache,target=/go/pkg/mod and--mount=type=cache,target=/root/.cache/go-build to theRUN command, we can use docker dedicated RUN cache to cache the dependencies and the compiled binary. Everytimes dockerBuilderKit run theRUN command, it will mount the cache to the specified target. This will leverage the artifacts from the previous build and save us a lot of time.
For instance, I made a change intogo.mod andrepository package and re-build the image. The output is as follows:

#9[builder 3/4] RUN--mount=type=cache,target=/go/pkg/mod--mount=type=bind,source=go.mod,target=go.mod--mount=type=bind,source=go.sum,target=go.sum     go mod download-x#9 DONE 0.3s#10[builder 4/4] RUN--mount=type=cache,target=/go/pkg/mod--mount=type=cache,target=/root/.cache/go-build--mount=type=bind,source=.,target=.CGO_ENABLED=0     go build-v-o /usr/local/bin/myapp ./#10 0.556 gotour/example/internal/repository#10 0.587 gotour/example/internal/service#10 0.597 gotour/example#10 DONE 1.0s

This log discloses that the dockerBuilderKit re-executedgo mod download command but it was exceptionally fast as the go mod reused the downloaded dependencies from the cache. Similarly, thego build command was also re-executed and remarkably swift, as it only recompiled three packages (repository,service andmain) instead of the whole project along with all Go standard libraries.

2.3. Multi platforms build (Bonus)

We can also use docker multi platform build to build the image for different platforms. For instance, we can build the image forlinux/amd64 andlinux/arm64 platforms.

#----- Builder stage -----#-FROM golang:1.21.3-alpine as builder+FROM --platform=${BUILDPLATFORM} golang:1.21.3-alpine as builder+ARG TARGETOS+ARG TARGETARCHRUN \   --mount=type=cache,target=/go/pkg/mod \   --mount=type=cache,target=/root/.cache/go-build \   --mount=type=bind,source=.,target=.\    CGO_ENABLED=0 \+   GOOS=${TARGETOS} \+   GOARCH=${TARGETARCH} \    go build -v -o /app/bin/main .

3. Conclusion

In this section, we have explored how to optimize the Dockerfile for a golang application. Key takeaways include leveraging Docker's multi-stage build to diminish image size (from >300Mb to 13Mb) and enhance security. Additionally, We delved into utilizing Docker's dedicated RUN cache to speed up the build process. However, this technique is primarily effective on the Local Development Environment. For the CI/CD pipeline, we need a more advanced technique to enable the cache across separate builds.

In the upcoming section, we will delve into configuring hot-reload in local development environment for a Golang application.

Thank you for reading.