Repository files navigation

This is the README for the Cog Git source tree:https://github.com/OpenSmalltalk/opensmalltalk-vm

Builds are tested automatically on each commit, for Windows, macOS, and Linux.Windows and Linux also have 32-bit targets. Squeak and Pharo VMs are built withand without JIT, with and without Spur, and with and without Sista, as availableper platform. All build artifacts remain accessible for 90 days for eachworkflow run; the latest artifacts are uploaded to a dedicated release tag(e.g., "latest-build"), overwritten with each successful run. If you wish tocommitwithout triggering a build, for example if you were to only edit thisdocumentation, then if you add[ci skip] somewhere in your commit message,no workflow run will be started for that commit.

We rely on source file substitutions in the src tree, specifically,any sq*SCCSVersion.h files anywhere in the tree are processed to replace$Rev$,$Date$, and$URL$ with the current revision (defined as thetimestamp %Y%m%d%H%M of the latest commit), the human readable date of thatcommit, and the url of the origin repository these sources were cloned from.

The first time you clone this repository, youmust therefore run this command:

./scripts/updateSCCSVersions

This will install filters, post-commit, and post-merge hooks to update thesq*SCCSVersion.h files anytime you commit or merge.

For easier use, we include the scripts/gitci and scripts/gitciplugins scripts tocommit changes to this branch and changes to the Cross and win32 plugins (whichare shared with the old Squeak interpreter trunk). If you decide not to usethese scripts for checking in sources, you should also run theupdateSCCSVersions script anytime you decide to usegit-reset orgit-checkout to make sure the stamps are updated correctly. Failing to do sowill result in incorrect version stamps in your compiled VMs.

• Overview
• Variants
• VM source directories
• Platform build directories
• Other directories (platforms, processors, deployment, image)

First, opensmalltalk-vm (a.k.a. the Cog VM) is the virtual machine beneath theCuis and Squeak Smalltalk dialects. For issues related to these systemsthat are unrelated to the VM itself, please use their forums:

• https://cuis.st/community
• https://squeak.org/community

Note that we dropped support for Newspeak and Pharo VM flavors in August 2025.The last working sources can be found in the tagslast-support-newspeak andlast-support-pharo.

Second, the core VM, which comprises the execution engine and garbage collector,and the core plugins, is developed in Smalltalk, using theVM Simulator. Thisrepository contains the code generated by the Simulator, and the platform supportcode for the entire VM, its CI infrastructure and so on. The core VMshouldnot be developed by editing the generated code. The core VM should bedeveloped using Smalltalk. The source code repository for the VM is
http://source.squeak.org/VMMaker.html.
You can find scripts to build a Smalltalk image in which to do core VM developmentin theimagedirectory in this repository. You can read about the Simulator here:

• https://www.researchgate.net/publication/328509577_Two_Decades_of_Smalltalk_VM_Development_Live_VM_Development_through_Simulation_Tools

Cog is an evolution of the Squeak Back-to-the-future Smalltalk virtual machinethat provides a number of different Smalltalk virtual machines. The VMs aredeveloped in Smalltalk, using all the dynamic and reflective facilities of theSqueak Smalltalk system. As such, developing in Cog is a delight. TheSmalltalk framework comprising the various Cog VMs is translated into C by itsSlang component to produce VM source that is combined with platform-specificsupport sources and compiled via a C compiler to obtain a fast production VM.This directory tree includes the output of Slang for various configurations of"Cog VM" along with the associated platform support code, plus build directories thatcan be used to produce production VMs.

This directory tree also includes a directory containing scripts that can be used tocreate an instance of the Smalltalk Cog development system, suitable for developingthe VM in Smalltalk, and for generating new VM sources. See the image subdirectory,described below.

The "Cog VM" comes in a bewildering variety of forms. The first distinctionis between Stack, Cog and Sista VMs. Stack VMs are those with context-to-stackmapping that optimise message sending by keeping method activations on a stackinstead of in contexts. These are pure interpreters but are significantlyfaster than the standard context-based Interpreter VM. Cog VMs add a JIT tothe mix, compiling methods used more than once to machine code on the fly.Sista VMs, as yet unrealised and in development, add support for adaptiveoptimization that does speculative inlining at the bytecode-to-bytecode level.

Another distinction is between "v3" VMs and Spur VMs. "v3" is the originalobject representation for Squeak as described in the back-to-the-future paper.Spur, as described on thewww.mirandabanda.org blog, and in the paper "A partial read barrier for efficient support of live object-oriented programming" by Miranda & Béra, is a faster objectrepresentation which uses generation scavenging, lazy forwarding for fastbecome, a single object header format common to 32 and 64 bit versions, and asegmented heap that can grow and shrink, releasing memory back to the host OS.Squeak 5.0, Cuis 5 and Pharo 5 and subsequent releases use Spur.

Another distinction is between normal single-threaded VMs that schedule "green"Smalltalk light-weight processes above a single-threaded VM, and multi-threadedVMs that share the VM between any number of native threads such that only onenative thread owns the VM at any one time, switching between threads on FFIcalls and callbacks or on Smalltalk process switches when Smalltalk processesare owned by threads. This architecture offers non-blocking FFI calls andinteroperability with multiple native threads, but does /not/ provide trueconcurrency. This multi-threaded support is as yet experimental.

The Slang output of the various VMs are kept in "vm source" directories. TheseC sources define the core VM (the Smalltalk execution engine and the memorymanager), and a substantial set of "plugins" that provide interfaces to variousexternal facilities via Smalltalk primitive methods. Each vm source directoryis specific to a particular VM, be it Squeak Cog Spur, or V3 Stack, etc.The plugins can be shared between VMs, choosing the set of plugins to includein a VM at build time (see plugins.int & plugins.ext in build directories).

The VM source are in directories such as

src/v3.sista- Smalltalk Sista V3src/spur32.sista- Smalltalk Sista Spursrc/spur32.cog - Smalltalk Cog Spursrc/spur64.cog - Smalltalk Cog Spur 64-bitsrc/spur32.stack - Smalltalk Stack Spursrc/spur64.stack - Smalltalk Stack Spur 64-bitsrc/v3.cog- Smalltalk Cog V3src/v3.stack - Smalltalk Stack V3

All plugins are in the directory

src/plugins

These contain many, but not all, of the plugins available for the VM. Otherscan be found in Cog, or in various Monticello packages in various repositories.

Each vm source directory contains several files, a subset of the following:

cogit.c- the JIT; a Cogit cooperates with a CoInterpreter.                          This simply includes a processor-specific JIT filecogitIA32.c et al   - relevant processor-specific JIT, selected by cogit.ccogit.h- the Cogit's API, as used by the CoInterpretercogmethod.h- the structure of a CogMethod, the output of the Cogitcointerp.c- the CoInterpreter's source filecointerp.h- the API of the CoInterpreter, as used by the Cogitcointerpmt.c- the multi-threaded CoInterpreterMT's source filecointerpmt.h- the API of the CoInterpreterMT, as used by the Cogitgcc3x-cointerp.c- cointerp.c massaged to interpret faster if using gccgcc3x-cointerpmt.c- ditto for cointerpmt.cgcc3x-interp.c- ditto for interp.cinterp.c- the StackInterpreter's source fileinterp.h- defines for the VM configuration, word size, etcvmCallback.h- the structure of the VM's VMCallbackContext

The current "official" build directories are of the formbuilding/OS_WordSize_Processor, and include

building/linux32x86   - uses autoconf, clang (or gcc), and makebuilding/linux64x64   - uses autoconf, clang (or gcc), and makebuilding/linux64ARMv8 - uses autoconf, clang (or gcc), and makebuilding/macos64x64  - macOS on Intel using clang and gmake (via XCode)building/macos64ARMv8 - macOS on ARM using clang and gmake (via XCode)building/win32x86     - uses msys2, clang, and make; supports Visual Studio Native Tools Command Promptbuilding/win64x64     - uses msys2, clang, and make; supports Visual Studio Native Tools Command Promptbuilding/win64ARMv8   - uses msys2, clang, and make; supports Visual Studio Native Tools Command Prompt

More can be added as required. In each there is a HowToBuild that describesthe necessary steps to compile a VM.

Within each building/OS_WordSize_Processor directory are a set of build directoriesfor specific configurations of Cog, and for support code and makefiles. Forexample, there exist

building/macos64x64/squeak.cog.spur   - A Cog JIT VM with Squeak branding,                                         using the Spur memory manager.building/macos64x64/squeak.stack.spur - A Stack interpreter VM with Squeak                                         branding, and the Spur memory manager.building/win32x86/squeak.cog.v3     - A Cog JIT VM with Squeak branding,                                         using the old Squeak memory manager.

etc.

There exist

    building/macos64x64/bochsx86 - Support libraries for the BochsIA32Plugin which                                is used to develop Cog itself.    building/macos64x64/bochsx64 - Support libraries for the BochsX64Plugin which                                is used to develop Cog itself.    building/macos64x64/gdbarm32 - Support libraries for the GdbARMPlugin which                                is used to develop Cog itself.    building/macos64x64/gdbarm64 - Support libraries for the GdbARMv8Plugin which                                is used to develop Cog itself.

and the intention is to add such directories to contain e.g. support code forthe Pharo Cairo and Freetype plugins, and anything else needed. By placingsupport directories in each build directory they can be shared between variousbranded VM builds, avoiding duplication.

There exist

building/macos32x86/common - Gnu Makefiles for building the various branded VMsbuilding/macos64x64/common - Gnu Makefiles for building the various branded VMsbuilding/win32x86/common   - Gnu Makefiles for building the various branded VMsbuilding/win64x64/common   - Gnu Makefiles for building the various branded VMs

And the intention is to add building/linuxNN????/common as soon as possible touse Gnu Makefiles to build all VMs on all platforms.

The scripts directory contains various scripts for validating and checking-ingenerated sources, packaging builds into installable artifacts (tar, msi, zip),and so on. The linux builds and the packaging scripts produce outputs in theproducts directory tree. The packaging scripts may choose to include Smalltalksource files included in the sources directory.

The platforms directory contains the associated platform-specific files thatcombine with the Slang-generated source to produce complete VMs. The structureis

platforms/Cross/vmplatforms/Cross/pluginsplatforms/iOS/vmplatforms/iOS/pluginsplatforms/unix/vm*platforms/unix/pluginsplatforms/win32/vmplatforms/win32/plugins

Each vm directory contains support for the core VM. Each plugin directorycontains run-time and build-time support for various plugins.

The processors directory contains the source for various processor simulators.The JIT is developed in Smalltalk by using one of these processor simulatorsto execute the code the JIT produces. Currently x86 & x86-64 are derived fromBochs, and ARMv6/v7 & ARMv8 are derived from gdb.

Customization of builds may be done in two main ways, by adapting an existingbuild directory, and by using facilities designed to be applied after the factto a preexisting build. Such facilities live in the top-level deploy directoryand are documented there; e.g. see deploy/packaging/README.win32.

Finally the image directory contains scripts that will build a "VMMaker" image,a Squeak Smalltalk image containing all the packages that comprise the Cogsystem, suitable for developing the VM and for generating (or updating) thesources in the vm source directories. There is also a script for generating a64-bit Spur image from a 32-bit Spur image, and the VMMaker.oscog package includescode for converting Spur images in either direction. Note that JIT development depends on processor simulations which are included under the top-level processors directory, Bochs providing x86 and x86-64, gdb providing ARMv6 and ARMv8. To do JIT development the processor plugins must be built. These will be built automatically if the simulator libraries are built. In several building top-level subdirectory are four directories, bochsx86, bochsx64, gdbarm32 & gdbarm64, which contain the build scripts. To build a VM suitable for full JIT simulation, first build the support code in the building subdirectory for your platform, and then build a squeak.cog.spur VM in the sibling directory.

Enjoy, and give forward.