Waiting for Postgres 18 – Docker Containers 34% Smaller

Posted byJeremy⋅April 7, 2025⋅Leave a comment

Filed Under containers,debian,devops,docker,kubernetes,packaging,size,technology

On February 25, 2025 Christoph Bergcommitted the patch:

Subject: [PATCH] Move JIT to new postgresql-18-jit package. (Closes: #927182)Make LLVM architectures a inclusion list so it works in the Architecture field.

This closedDebian bug 927182 which had been opened in April 2019 by Laurence Parry. That bug had raised concerns over the significant size increase of adding LLVM as a requirement to support Postgres JIT functionality.

Postgres supports packaging LLVM as a separate optional package without needing to recompile database binaries. Postgres is compiled once, and it performs a runtime check whether LLVM libraries are present. It gracefullydisables JIT functionality in the database if LLVM libraries are not installed.

From: Andres FreundSubject: Re: llvm dependency and space concernsDate: 2025-01-12 00:03:43Lists: pgsql-hackersHi,On 2025-01-11 13:22:39 -0800, Jeremy Schneider wrote:> It's a cleaner solution if JIT works more like an extension, and we can> run a single build and split JIT into a separate package.It does work like that. Only llvmjit.so has the llvm dependency, the mainpostgres binary doesn't link to llvm. If llvmjit.so isn't available, jit issilently disabled.Andres

In the official community Postgres RPM/yum repository LLVM was split out to a separate package – but it had never been prioritized in the official community Debian repositories to make these code updates until now.

Christophnoted on the mailing lists that the work is not yet finished – there is still a little more that needs to be done:

From: Christoph BergCc: pgsql-pkg-debian@lists.postgresql.orgSubject: Re: pg18 patch: separate package for llvm/jitDate: Fri, 28 Feb 2025 16:02:42 +0100Re: Jeremy Schneider> I would like to propose this change for Postgres 18.  Hi,I committed a change to the PG18 packaging that implements that split.The new package is called postgresql-18-jit.There is more work to do on each of the extension packages currentlydepending on "postgresql-18-jit-llvm (= llvmversion)", that needs to beconverted to "Breaks: postgresql-18-jit-llvm (<< llvmversion)". (Thisis also the reason the jit package is not called like that because theversion number there is not the PG version number.)This will likely happen when extensions are moved to PG18 inSeptember.I'm unsure if the split should be backported to PG 17 and earliersince it will affect production systems in some way.Christoph

CloudNativePG running on Kubernetes is a fully automated control plane that uses container images and keeps your Postgres databases running reliably, backed up, simple to patch/update and able to recover on their own if something goes wrong. Theofficial container images provided by CloudNativePG directly use the official “PGDG” Postgres community Debian packages on top of the official docker-provided “slim” debian base image. This has the benefit that CNPG users are part of the “herd” using the same binaries which are used by Debian Postgres users globally; we’re not going to hit any corner cases from compilation flags or configure options that aren’t widely used. Another commonly used container is theDocker “official image” that you get when you typedocker run postgres – these images also directly use the official Postgres community Debian packages by default.

Generally speaking, the size of docker containers can vary widely. Minimizing dependencies and reducing container size has many benefits: less stuff to patch and update over time, fewer things that can get CVEs, fewer unnecessary utilities that can be leveraged by attackers, faster build times, less disk space usage, reduced network consumption and faster startup times in environments where containers are often migrated and rescheduled across fleets (moving to new hardware where those containers had never previously run, and thus images need to be downloaded).

Let’s take a look at the sizes of common Postgres containers:

# docker imagesREPOSITORY                              TAG                            SIZEghcr.io/cloudnative-pg/postgresql       17.4-standard-bookworm         641MBghcr.io/cloudnative-pg/postgresql       17.4-minimal-bookworm          413MBpostgres                                17.4-bookworm                  438MBpostgres                                17.4-alpine3.21                278MB

Docker provides an official Postgres image based on Alpine (a Linux distribution designed to be small, simple and secure). Postgres is built from source for the Alpine container. Alpine uses the musl C library which doesn’t have the same adoption level in the general public and test coverage in the postgres community build farm as the GNU C library. Nonetheless, we see above that Docker’s official Alpine postgres image is 36% smaller than Docker’s official Debian Bookworm image. (From 438MB down to 278MB.)

Interestingly, LLVM alone seems responsible for half of the Alpine image’s size!

# docker run -it postgres:17.4-alpine3.21 apk info llvm19-libsWARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.21/main: No such file or directoryWARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.21/community: No such file or directoryllvm19-libs-19.1.4-r0 description:LLVM 19 runtime libraryllvm19-libs-19.1.4-r0 webpage:https://llvm.org/llvm19-libs-19.1.4-r0 installed size:153 MiB

The Alpine build might be useful for use cases where every byte matters – but it’s lacking in one of Postgres greatest strengths:extensibility. Another significant advantage of leveraging the community Debian packages is that we get free access to all of the Postgres Extensions that are already packaged for Debian. Adding a supported extension to a container isa one-liner change to the Dockerfile!

CloudNativePG builds and provides their own images on the Github Container Registry. At present there are two flavors of CNPG images: “minimal” and “standard”.

The CNPG minimal image is 413MB – about the same size as the docker image without any extensions or extras.

The CNPG standard image 641MB and includes three default extensions (pgvector, pgaudit and pg-failover-slots) as well as all Debian-supported glibc locales. The locales are responsible for 99% of the size increase.

# docker run -it ghcr.io/cloudnative-pg/postgresql:17.4-standard-bookworm dpkg-query --show --showformat='${Package}\t${Installed-Size} KB\n' locales-all postgresql-17-pgvector postgresql-17-pgaudit postgresql-17-pg-failover-slotslocales-all     227366 KBpostgresql-17-pg-failover-slots 101 KBpostgresql-17-pgaudit   106 KBpostgresql-17-pgvector  767 KB

Most users should not need these Debian glibc locales, especially since these images include ICU (which interestingly is much smaller than the Debianlocales-all package). Most of us should be starting with the minimal image and customizing it with only the extensions we need.

But what about LLVM? Christoph’s commit should make JIT an optional install instead of a required install; how will this impact the minimal image size?

It took mea little work to get the syntax right, but it turns out this wasn’t too hard to test by tweaking the CNPG Dockerfile to use the official postgres community pg18 debian package snapshots.

https://github.com/ardentperf/postgres-containers/commit/6388113b36824e3a88f47faabf0ddd386b96e41a

Author: Jeremy Schneider <schneider@ardentperf.com>Date:   Sun Apr 6 21:58:45 2025 -0700    test pg18 development snapshots https://wiki.postgresql.org/wiki/Apt/FAQ#Development_snapshotsdiff --git a/Dockerfile b/Dockerfileindex d4f02b7..8a03349 100644--- a/Dockerfile+++ b/Dockerfile@@ -8,7 +8,12 @@ ENV PATH=$PATH:/usr/lib/postgresql/$PG_MAJOR/bin RUN apt-get update && \     apt-get install -y --no-install-recommends postgresql-common ca-certificates gnupg && \-    /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y && \+    true++COPY pg18snapshot.pref /etc/apt/preferences.d/+COPY pg18snapshot.list /etc/apt/sources.list.d/++RUN apt-get update && \     apt-get install -y --no-install-recommends -o Dpkg::::="--force-confdef" -o Dpkg::::="--force-confold" postgresql-common && \     sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf && \     apt-get install -y --no-install-recommends \diff --git a/pg18snapshot.list b/pg18snapshot.list+deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] http://apt.postgresql.org/pub/repos/apt/ bookworm-pgdg-snapshot main 18diff --git a/pg18snapshot.pref b/pg18snapshot.pref+Package: *+Pin: origin apt.postgresql.org+Pin-Priority: 1001

First I built version 17.4 locally to confirm that I would get the same size container as what’s in the official registry. Then I used the modified Dockerfile to build version 18.

# git checkout mainSwitched to branch 'main'Your branch is up to date with 'origin/main'.# docker build . --target minimal --tag minimal17 --build-arg PG_VERSION=17.4[+] Building 0.1s (7/7) FINISHED                                                                                      docker:default => [internal] load build definition from Dockerfile                                                                            0.0s => => transferring dockerfile: 1.35kB                                                                                          0.0s => [internal] load metadata for docker.io/library/debian:bookworm-slim                                                         0.0s => [internal] load .dockerignore                                                                                               0.0s => => transferring context: 2B                                                                                                 0.0s => [minimal 1/3] FROM docker.io/library/debian:bookworm-slim                                                                   0.0s => CACHED [minimal 2/3] RUN apt-get update &&     apt-get install -y --no-install-recommends postgresql-common ca-certificate  0.0s => CACHED [minimal 3/3] RUN usermod -u 26 postgres                                                                             0.0s => exporting to image                                                                                                          0.0s => => exporting layers                                                                                                         0.0s => => writing image sha256:d2d9baf4d2384dbfe58cd14bd0adc5aaca2beff95a4a7e37712ddc26f14a3a7d                                    0.0s => => naming to docker.io/library/minimal17                                                                                    0.0s# git checkout test18Switched to branch 'test18'Your branch is up to date with 'ardent/test18'.# docker build . --target minimal --tag minimal18 --build-arg PG_VERSION=18[+] Building 0.1s (11/11) FINISHED                                                                                    docker:default => [internal] load build definition from Dockerfile                                                                            0.0s => => transferring dockerfile: 1.42kB                                                                                          0.0s => [internal] load metadata for docker.io/library/debian:bookworm-slim                                                         0.0s => [internal] load .dockerignore                                                                                               0.0s => => transferring context: 2B                                                                                                 0.0s => [minimal 1/6] FROM docker.io/library/debian:bookworm-slim                                                                   0.0s => [internal] load build context                                                                                               0.0s => => transferring context: 298B                                                                                               0.0s => CACHED [minimal 2/6] RUN apt-get update &&     apt-get install -y --no-install-recommends postgresql-common ca-certificate  0.0s => CACHED [minimal 3/6] COPY pg18snapshot.pref /etc/apt/preferences.d/                                                         0.0s => CACHED [minimal 4/6] COPY pg18snapshot.list /etc/apt/sources.list.d/                                                        0.0s => CACHED [minimal 5/6] RUN apt-get update &&     apt-get install -y --no-install-recommends -o Dpkg::::="--force-confdef" -o  0.0s => CACHED [minimal 6/6] RUN usermod -u 26 postgres                                                                             0.0s => exporting to image                                                                                                          0.0s => => exporting layers                                                                                                         0.0s => => writing image sha256:c2471c9180750c1983177693ef21b609443c4b5aa89f9f5ffe9cb8e159ebe00d                                    0.0s => => naming to docker.io/library/minimal18                                                                                    0.0s# docker imagesREPOSITORY                              TAG                            SIZEminimal18                               latest                         262MBminimal17                               latest                         413MBghcr.io/cloudnative-pg/postgresql       17.4-standard-bookworm         641MBghcr.io/cloudnative-pg/postgresql       17.4-minimal-bookworm          413MBpostgres                                17.4-bookworm                  438MBpostgres                                17.4-alpine3.21                278MB

The minimal CNPG image for Postgres 18 is 262MB – that’s 34% smaller than the minimal CNPG image for Postgres 17 of 413MB.

This is great news – another reason to look forward to Postgres 18!

About Jeremy

Building and running reliable data platforms that scale and perform.about.me/jeremy_schneider

View all posts by Jeremy»

« Testing loadBalancerSourceRanges with CloudNativePG on Azure Kubernetes

Collation Torture Test versus Debian»

Discussion

No comments yet.

Leave a New CommentCancel reply

This site uses Akismet to reduce spam.Learn how your comment data is processed.

Disclaimer

This is my personal website. The views expressed here are mine alone and may not reflect the views of my employer.I am currently looking for consulting and/or contracting work in the USA around the oracle database ecosystem.

contact:312-725-9249 orschneider @ ardentperf.com