Last year, we started switching our development environment atSpareRoom from aVagrant/VirtualBox solution toDocker. This was to make a more robust environment that supported all our web-app territories at the same time, as well as support theApple Silicon Macs we'd be upgrading to.

Docker Desktop on MacOS

While the Docker environment had many advantages, the performance on our MacOS dev machines came as a shock, and we quickly found out this was due to the dreadful file system performance (mainly for Docker volumes). The effect was very obvious, e.g. an Apache thread would take an extra couple of seconds just to read the app libraries. We were looking into implementing aMutagen solution for syncing local/docker filesystems, which requires some setup but has helped other MacOS users considerably, when Docker Desktop v4.6 was released (March 2022) with an interesting new feature.

VirtioFS

Docker Desktop 4.6 added beta support forVirtioFS forMacOS Monterey and above. Just enabling it in the (beta at the time) settings screen and relaunching the containers was all needed for instant performance gains!

To be honest, the first time I enabled it, Docker Desktop simply got stuck and had to be killed. The Docker engine would not start (shown as "stopped") until I disabled the option directly in the settings file:

~/Library/Group\ Containers/group.com.docker/settings.json

(setuseVirtualizationFrameworkVirtioFS to false)
But the culprit was soon found - I had migrated to a new Mac and in my Docker file sharing settings a no-longer existing directory was listed. If you have a similar problem, check in the above settings file in case there is a removed or inaccessible directory underfilesharingDirectories. Apparently VirtioFS just hangs forever without errors in that case - yeah, I know, not the best programming practice around that!

On Docker Desktop v4.15, a bug was fixed where some users had file permission issues, but with it the VirtioFS performance degraded significantly. The current v4.17 is a faster than v4.15/4.16, but my advice is tryv4.14.1, which is the latest "fast" version. If you have no issues with file permissions (it was not a widespread problem anyway), keep that until the performance regression is fixed.

Colima

As I was tracking the above performance regression, I discovered an open-source container runtime for MacOS/Linux calledColima. It is pretty much a Docker Desktop drop-in replacement and also offers VirtioFS (onVentura - i.e. MacOS 13.x or newer). You don't need to uninstall Docker Desktop, just stop it and install Colima e.g. throughHomebrew:

brew install colima

You should already have docker installed (otherwisebrew install docker would suffice, and you create a VM to run the containers with the following command:

colima start --cpu 2 --memory 2 --vm-type=vz --mount-type=virtiofs

You can specify the CPU/RAM you prefer, and if you need a Rosetta 2-enabled vm, the option is--vz-rosetta. On the first run it will setup the VM to run your containers. After it finishes, thedocker context show command should returncolima, which means docker runs under Colima and you can use docker as usual under it.

The commandcolima stop will returndocker context show todefault, so starting Docker Desktop after that allows you to go back to it if you wish.

Colima seems to be even faster than Docker Desktop 4.14, however, I did encounter some stability issues - the project does specify that the vz/VirtioFS support is recent, so hopefully it will quickly mature. It is definitely worth trying out and keeping an eye on, especially since it is so simple to set up as an alternative.

Performance Comparison

I ran a couple of relatively file-access heavy comparison tests on the latest Docker Desktop for both "standard"gRPC-FUSE andVirtioFS, as well as the earlier4.14.1 version and the latestColima.

The first test is a collection of SpareRoom API tests without any module preloading, that have to load parts of the webapp and also access the mysql and sphinx search containers. The second is the test suite of theMoose 2.2203 Perl package, which is my go-to when I want an "overweight OO framework" test.
Note that these are far from pure file I/O tests: a file I/O-only test would show much more pronounced differences, but I don't like synthetic tests. Hence, I went as real-world as possible, for example the Moose test suite runs like that whenever you install the package withcpanm Moose.

Here are the results, running on anM1 Pro (arm64 containers):

VirtioFS was a game-changer, it is 2-3x faster in these tests compared to gRPC-FUSE, and I've seen even bigger differences than that. But, given that the Moose test is twice as fast running natively on the same M1 Pro (39s), there could still be improvements. Especially for Docker Desktop which is currently lagging not only behind Colima, but also from its own earlier versions. Colima on the other hand, while measurably faster does seem a bit immature - I've had to restart my environment due to issues etc. But given the fact the VirtioFS support is recent and under active development, it's definitely one to watch.