Repository files navigation

Esperanto makes OCaml a build-once run-anywhere language, like itself, exceptit doesn't need an interpreter or virtual machine. Indeed, OCaml is able toproduce anative executable which requires few libraries:

• a standard C library
• libasmrun.a (thecaml-runtime)
• unix.cmxa andthreads.cmxa

Esperanto replaces the host's C library byCosmopolitan. Then,it outputs a POSIX-approved polyglot format that runs on many platforms. Formore details, please read theαcτµαlly pδrταblε εxεcµταblε.

The goal of esperanto is to provide atoolchain which is able to compile anOCaml project toaarch64 andx86_64 and link these artifacts together toproduce an actually portable executable. Esperanto is primarily a distributiontool rather than a development tool. We advise the user to have a finalizedproject and then use Esperanto to produce a distribution.

The pipeline used to produce such a portable executable must respect certainconstraints:

• not depend on dynamic objects (*.so)
• the C files required for compilation can also be compiled with theCosmopolitan C library

Let's take the example of a simple program to understand the basic "pipeline"required to produce a.com (for COsMpolitan) executable. We recommend usingEsperanto in a [Docker][docker] or "clean" system (like a special[opam switch][switch]). Once again, the aim is to produce an artifact from afinalized project, not to develop an application. So you need opam. Let's takeDocker as an example:

$ docker run -it --platform amd64 ocaml/opam:ubuntu-23-04-ocaml-4.14$ opam pin add -y https://github.com/dinosaure/esperanto.git

Esperanto installs 4 packages:

• conf-cosmopolitan, this package verifies that your system can, indeed, runape programs.
• esperanto-cosmopolitan, this package is the Cosmopolitan toolchain asavailable from [cosmocc.zip][cosmo.zip] (v3.1.3) packaged foropam. Itverifies the integrity of distribution files according to these[fingerprints][./toolchain/cosmocc.digest]

Finally, 2 versions of the OCaml compiler are available:

• To produce anx86_64 program (x86_64-esperanto)
• To produce anaarch64 program (aarch64-esperanto)

Let's start to make a simpleHello World! example:

$ cat>main.ml<<EOFlet () = print_endline "Hello World!"EOF$ ocamlfind -toolchain x86_64_esperanto opt -o main.x86_64.elf main.ml$ ocamlfind -toolchain aarch64_esperanto opt -o main.aarch64.elf main.ml$ apelink -o main.com \     -l$(opam var bin)/ape-x86_64.elf \     -l$(opam var bin)/ape-aarch64.elf \     -M$(opam var bin)/ape-m1.c \     main.x86_64.elf main.aarch64.elf$ sh -c"./main.com"Hello World!

Producing an executable from your OCaml project that is portable on all systemsrequires 2 precise points:

1. project dependencies must be available to compile in a consistent context.Such consistency can be achieved bydune and a"monorepo" format for yourproject.
2. C dependencies must be able to compile with Cosmopolitan.[cosmocc.zip][cosmocc.zip] offers a patched version of GCC and Cosmopolitanexposes many functions. However, you may need to patch your C code (inparticular, you can use__COSMOPOLITAN__).

If these 2 points are met (getting the dependency sources and havingCosmopolitan-compatible C code), we're ready to produce the final artifact usingthe Esperanto toolchain. So, 3 steps are necessary:

1. compile project for x86_64 architecture
2. compile project for aarch64 architecture
3. link the 2ELF files into a portable binary

To achieve our 2 objectives, we're going to usedune to provide us with thefamous contexts linked to our 2 toolchains (x86_64 andaarch64) andopam monorepo to download locally the dependencies needed for our project (theconstraint implies that all dependencies use dune). Let's play with [hxd][hxd],a small project.

You can take your favorite project and compile executables with theEsperanto toolchain. You need to specify two new contexts:

$ git clone https://github.com/dinosaure/hxd.git$cd hxd$ opam repo add opam-overlays https://github.com/dune-universe/opam-overlays.git$ sudo apt install pkg-config$ cat>dune-workspace<<EOF(lang dune 2.0)(context (default))(context (default  (name x86_64-esperanto)  (toolchain x86_64_esperanto)  (host default)))(context (default  (name aarch64-esperanto)  (toolchain aarch64_esperanto)  (host default)))EOF

Finally, as a static program, you must fetch dependencies withopam monorepo and build your project with them:

$ opam monorepo lock --ocaml-version=4.14.1$ opam monorepo pull$ dune build bin/xxd.exe$ apelink -o xxd.com \     -l$(opam var bin)/ape-x86_64.elf \     -l$(opam var bin)/ape-aarch64.elf \     -M$(opam var bin)/ape-m1.c \     _build/bin/x86_64-esperanto/xxd.exe.dbg \     _build/bin/aarch64-esperanto/xxd.exe.dbg

Now, you have a portable/polyglot program! The user canassimilate theprogram with the--assimilate option. Assimilation consists in "specializing"the executable for the host system.

$ sh -c"./bin/xxd.com --assimilate"$ uname -px86_64$ file bin/xxd.combin/xxd.com: ELF 64-bit LSB executable, x86-64

The--assimilate option modify the executable itself to becomereallynative to your platform - in other words, the executable no longer becomespolyglot!

Such design comes fromMirageOS andSolo5. For C stubs, thecosmocc toolchain provides the__COSMOPOLITAN__ definition which permits toorchestrate your compilation of C files according to the OCaml toolchain.

Currently, we support OCaml 4.14.

Currently,zsh < 5.9.0 does not support well Cosmopolitan/APE binaries.However, the recent version fix the initial issue. In the case of you havezsh < 5.9.0, you can use the--assimilate option to modify the executableto areal native application:

$ zsh --versionzsh 5.8.1$ zsh$ ./a.outzsh:exec format error: ./a.out$ sh -c"./a.out --assimilate"$ ./a.outHello World!

If the issue persist with Pythonsubprocess or old versions of fish, we alsoadvise an "assimilation" in these cases. However, as explained above, the binaryno longer becomespolyglot!

An other issue on Linux is aboutbinfmt_misc which wants to interpret thebinary as a Windows executable (due to the header of the APE binary). A goodexplanation is available here:jart/cosmopolitan#2