Repository files navigation

Binaryen is a compiler and toolchain infrastructure library for WebAssembly,written in C++. It aims to makecompiling to WebAssemblyeasy, fast, andeffective:

• Easy: Binaryen has a simpleC API in a single header, and can also beused from JavaScript. It accepts input inWebAssembly-likeform but also accepts a generalcontrol flow graph forcompilers that prefer that.

• Fast: Binaryen's internal IR uses compact data structures and is designedfor completely parallel codegen and optimization, using all available CPUcores. Binaryen's IR also compiles down to WebAssembly extremely easily andquickly because it is essentially a subset of WebAssembly.

• Effective: Binaryen's optimizer has many passes (see an overview laterdown) that can improve code size and speed. These optimizations aim to makeBinaryen powerful enough to be used as acompiler backend byitself. One specific area of focus is on WebAssembly-specific optimizations(that general-purpose compilers might not do), which you can think of aswasmminification, similar to minification for JavaScript, CSS, etc., allof which are language-specific.

Toolchains using Binaryen as acomponent (typically runningwasm-opt) include:

• Emscripten (C/C++)
• wasm-pack (Rust)
• J2CL (Java;J2Wasm)
• Kotlin (Kotlin/Wasm)
• Dart (Flutter)
• wasm_of_ocaml (OCaml)

For more on how some of those work, see the toolchain architecture parts oftheV8 WasmGC porting blogpost.

Compilers using Binaryen as alibrary include:

• AssemblyScript which compiles a variant of TypeScript to WebAssembly
• wasm2js which compiles WebAssembly to JS
• Asterius which compiles Haskell to WebAssembly
• Grain which compiles Grain to WebAssembly

Binaryen also provides a set oftoolchain utilities that can

• Parse andemit WebAssembly. In particular this lets you loadWebAssembly, optimize it using Binaryen, and re-emit it, thus implementing awasm-to-wasm optimizer in a single command.
• Interpret WebAssembly as well as run the WebAssembly spec tests.
• Integrate withEmscripten in order to provide acomplete compiler toolchain from C and C++ to WebAssembly.
• Polyfill WebAssembly by running it in the interpreter compiled toJavaScript, if the browser does not yet have native support (useful fortesting).

Consult thecontributing instructions if you're interested inparticipating.

Binaryen's internal IR is designed to be

• Flexible and fast for optimization.
• As close as possible to WebAssembly so it is simple and fast to convertit to and from WebAssembly.

There are a few differences between Binaryen IR and the WebAssembly language:

• Tree structure

  • Binaryen IRis a tree, i.e., it has hierarchical structure,for convenience of optimization. This differs from the WebAssembly binaryformat which is a stack machine.

  • Binaryen uses Stack IR to optimize "stacky" code (that can't berepresented in structured form).

  • When stacky code must be represented in Binaryen IR, such as withmultivalue instructions and blocks, it is represented with tuple types thatdo not exist in the WebAssembly language. In addition to multivalueinstructions, locals and globals can also have tuple types in Binaryen IRbut not in WebAssembly. Experiments show that better support formultivalue could enable useful but small code size savings of 1-3%, so ithas not been worth changing the core IR structure to support it better.

  • Block input values (currently only supported incatch blocks in theexception handling feature) are represented aspop subexpressions.

• Types and unreachable code

  • WebAssembly limits block/if/loop types to none and the concrete value types(i32, i64, f32, f64). Binaryen IR has an unreachable type, and it allowsblock/if/loop to take it, allowinglocal transforms that don't need toknow the global context. As a result, Binaryen's defaulttext output is not necessarily valid wasm text. (To get valid wasm text,you can do--generate-stack-ir --print-stack-ir, which prints Stack IR,this is guaranteed to be valid for wasm parsers.)

  • Binaryen supports astringref type. This is similar to the currently-inactivestringref proposal, with the difference that the string type is asubtype ofexternref rather thananyref. Doing so allows toolchains toemit code in a form that usesjs string builtins which Binaryen can then"lift" into stringref in its internal IR, optimize (for example, aconcatenation of "a" and "b" can be optimized at compile time to "ab"), andthen "lower" that into js string builtins once more.

• Blocks

  • Binaryen IR has only one control flow structure that contains avariable-length list of children: the block. WebAssembly on the other handallows all control flow structures, such as loops, if arms, and functionbodies, to have multiple children. In Binaryen IR, these other control flowstructures have a single child. This child may of course be a block. Themotivation for this property is that many passes need special code foriterating on lists of instructions, so having a single IR node with a listsimplifies them.

  • As in the Wasm text format, blocks and loops may have names. Branch targetsin the IR are resolved by name (as opposed to nesting depth). This has 2consequences:

    • Blocks without names may not be branch targets.

    • Names are required to be unique. (Reading .wat files with duplicate namesis supported; the names are modified when the IR is constructed).

  • As an optimization, a block with no name, which can never be a branchtarget, will not be emitted when generating wasm. Instead its list ofchildren will be directly used in the containing control flow structure.Such a block is sometimes called an "implicit block".

• Reference Types

  • The wasm text and binary formats require that a function whose address istaken byref.func must be either in the table, or declared via an(elem declare func $..). Binaryen will emit that data when necessary, butit does not represent it in IR. That is, IR can be worked on without needingto think about declaring function references.

  • Binaryen IR allows non-nullable locals in the form that the Wasm spec does,in which alocal.get must be structurally dominated by alocal.set inorder to validate (that ensures we do not read the default value of null).Despite being aligned with the Wasm spec, there are some minor details thatyou may notice:

    • A namelessBlock in Binaryen IR does not interfere with validation.Nameless blocks are never emitted into the binary format (we just emittheir contents), so we ignore them for purposes of validatingnon-nullable locals. As a result, if you read wasm text emitted byBinaryen then you may see what seems to be code that should not validateper the spec (and may not validate in Wasm text parsers), but thatdifference will not exist in the binary format (binaries emitted byBinaryen will always work everywhere, aside from bugs of course).

    • The Binaryen pass runner will automatically fix up validation after eachpass (finding things that do not validate and fixing them up, usually bydemoting a local to be nullable). As a result you do not need to worrymuch about this when writing Binaryen passes. For more details see therequiresNonNullableLocalFixups() hook inpass.h and theLocalStructuralDominance class.

  • Binaryen IR uses the most refined types possible for references,specifically:

    • The IR type of aref.func is always an exact, non-nullable reference toa defined function type, and not plainfuncref, even if no featuresbeyond basic reference types are enabled.

    • The IR type of allocation instructions such asstruct.new orarray.new is always an exact reference, even if Custom Descriptors arenot enabled.

    • Non-nullable types are also used for the type thattry_table sendson branches (if we branch, a null is never sent), that is, it sends(ref exn) and not (ref null exn).

    • As a result, non-nullable and exact references are generally allowed inthe IR even when GC or Custom Descriptors is not enabled. When reading abinary, the more refined types will be applied as we build the IR.

    In all cases the binary writer will generalize the type as necessary forthe enabled feature set. For example, if only Reference Types is enabled,all function reference types will be emitted asfuncref.

  • br_if output types are more refined in Binaryen IR: they have the type ofthe sent value operand, when it exists. In the Wasm spec the type is thatof the branch target, which may be less refined. Using the more refinedtype here ensures that we optimize in the best way possible, using all thetype information, but it does mean that some roundtripping operations maylook a little different. In particular, when we emit abr_if whose typeis more refined in Binaryen IR, then we emit a cast right after it torecover the more refined type. That may cause a few bytes of extra size inrare cases (we avoid this overhead in the common case where thebr_ifvalue is unused).

As a result, you might notice that round-trip conversions (wasm => Binaryen IR=> wasm) change code a little in some corner cases.

• When optimizing Binaryen uses an additional IR, Stack IR (seesrc/wasm-stack.h). Stack IR allows a bunch of optimizations that aretailored for the stack machine form of WebAssembly's binary format (but StackIR is less efficient for general optimizations than the main Binaryen IR). Ifyou have a wasm file that has been particularly well-optimized, a simpleround-trip conversion (just read and write, without optimization) may causemore noticeable differences, as Binaryen fits it into Binaryen IR's morestructured format. If you also optimize during the round-trip conversion thenStack IR opts will be run and the final wasm will be better optimized.

Notes when working with Binaryen IR:

• As mentioned above, Binaryen IR has a tree structure. As a result, eachexpression should have exactly one parent - you should not "reuse" a node byhaving it appear more than once in the tree. The motivation for thislimitation is that when we optimize we modify nodes, so if they appear morethan once in the tree, a change in one place can appear in anotherincorrectly.
• For similar reasons, nodes should not appear in more than one functions.

Binaryen intrinsic functions look like calls to imports, e.g.,

(import"binaryen-intrinsics""foo" (func$foo))

Implementing them that way allows them to be read and written by other tools,and it avoids confusing errors on a binary format error that could happen inthose tools if we had a custom binary format extension.

An intrinsic method may be optimized away by the optimizer. If it is not, itmust belowered before shipping the wasm, as otherwise it will look like acall to an import that does not exist (and VMs will show an error on not havinga proper value for that import). That final lowering isnot doneautomatically. A user of intrinsics must run the pass for that explicitly,because the tools do not know when the user intends to finish optimizing, as theuser may have a pipeline of multiple optimization steps, or may be doing localexperimentation, or fuzzing/reducing, etc. Only the user knows when the finaloptimization happens before the wasm is "final" and ready to be shipped. Notethat, in general, some additional optimizations may be possible after the finallowering, and so a useful pattern is to optimize once normally with intrinsics,then lower them away, then optimize after that, e.g.:

wasm-opt input.wasm -o output.wasm -O --intrinsic-lowering -O

Each intrinsic defines its semantics, which includes what the optimizer isallowed to do with it and what the final lowering will turn it to. Seeintrinsics.hfor the detailed definitions. A quick summary appears here:

• call.without.effects: Similar to acall_ref in that it receivesparameters, and a reference to a function to call, and calls that functionwith those parameters, except that the optimizer can assume the call has noside effects, and may be able to optimize it out (if it does not have aresult that is used, generally).

This repository contains code that builds the following tools inbin/ (see thebuilding instructions):

• wasm-opt: Loads WebAssembly and runs Binaryen IR passes on it.
• wasm-as: Assembles WebAssembly in text format (currently S-Expressionformat) into binary format (going through Binaryen IR).
• wasm-dis: Un-assembles WebAssembly in binary format into text format(going through Binaryen IR).
• wasm2js: A WebAssembly-to-JS compiler. This is used by Emscripten togenerate JavaScript as an alternative to WebAssembly.
• wasm-reduce: A testcase reducer for WebAssembly files. Given a wasm filethat is interesting for some reason (say, it crashes a specific VM),wasm-reduce can find a smaller wasm file that has the same property, which isoften easier to debug. See thedocsfor more details.
• wasm-shell: A shell that can load and interpret WebAssembly code. It canalso run the spec test suite.
• wasm-emscripten-finalize: Takes a wasm binary produced by llvm+lld andperforms emscripten-specific passes over it.
• wasm-ctor-eval: A tool that can execute functions (or parts of functions)at compile time.
• wasm-merge: Merges multiple wasm files into a single file, connectingcorresponding imports to exports as it does so. Like a bundler for JS, butfor wasm.
• wasm-metadce: A tool to remove parts of Wasm files in a flexible waythat depends on how the module is used.
• binaryen.js: A standalone JavaScript library that exposes Binaryen methods forcreating and optimizing Wasm modules. For builds, seebinaryen.js on npm (or download it directly fromGitHub orunpkg). Minimal requirements: Node.js v15.8 or Chrome v75 or Firefox v78.

All of the Binaryen tools are deterministic, that is, given the same inputs you should always get the same outputs. (If you see a case that behaves otherwise, please file an issue.)

Usage instructions for each are below.

Binaryen containsa lot of optimization passesto make WebAssembly smaller and faster. You can run the Binaryen optimizer byusingwasm-opt, but also they can be run while using other tools, likewasm2js andwasm-metadce.

• The default optimization pipeline is set up by functions likeaddDefaultFunctionOptimizationPasses.
• There are variouspass optionsthat you can set, to adjust the optimization and shrink levels, whether toignore unlikely traps, inlining heuristics, fast-math, and so forth. Seewasm-opt --help for how to set them and other details.

See each optimization pass for details of what it does, but here is a quickoverview of some of the relevant ones:

• CoalesceLocals - Key "register allocation" pass. Does a live rangeanalysis and then reuses locals in order to minimize their number, as well asto remove copies between them.
• CodeFolding - Avoids duplicate code by merging it (e.g. if twoif armshave some shared instructions at their end).
• CodePushing - "Pushes" code forward past branch operations, potentiallyallowing the code to not be run if the branch is taken.
• DeadArgumentElimination - LTO pass to remove arguments to a function if itis always called with the same constants.
• DeadCodeElimination
• Directize - Turn an indirect call into a normal call, when the table indexis constant.
• DuplicateFunctionElimination - LTO pass.
• Inlining - LTO pass.
• LocalCSE - Simple local common subexpression elimination.
• LoopInvariantCodeMotion
• MemoryPacking - Key "optimize data segments" pass that combines segments,removes unneeded parts, etc.
• MergeBlocks - Merge ablock to an outer one where possible, reducingtheir number.
• MergeLocals - When two locals have the same value in part of theiroverlap, pick in a way to help CoalesceLocals do better later (split off fromCoalesceLocals to keep the latter simple).
• MinifyImportsAndExports - Minifies them to "a", "b", etc.
• OptimizeAddedConstants - Optimize a load/store with an added constant intoa constant offset.
• OptimizeInstructions - Key peephole optimization pass with a constantlyincreasing list of patterns.
• PickLoadSigns - Adjust whether a load is signed or unsigned in order toavoid sign/unsign operations later.
• Precompute - Calculates constant expressions at compile time, using thebuilt-in interpreter (which is guaranteed to be able to handle any constantexpression).
• ReReloop - Transforms wasm structured control flow to a CFG and then goesback to structured form using the Relooper algorithm, which may find moreoptimal shapes.
• RedundantSetElimination - Removes alocal.set of a value that is alreadypresent in a local. (Overlaps with CoalesceLocals; this achieves the specificoperation just mentioned without all the other work CoalesceLocals does, andtherefore is useful in other places in the optimization pipeline.)
• RemoveUnsedBrs - Key "minor control flow optimizations" pass, includingjump threading and various transforms that can get rid of abr orbr_table(like turning ablock with abr in the middle into anif when possible).
• RemoveUnusedModuleElements - "Global DCE", an LTO pass that removesimports, functions, globals, etc., when they are not used.
• ReorderFunctions - Put more-called functions first, potentially allowingthe LEB emitted to call them to be smaller (in a very large program).
• ReorderLocals - Put more-used locals first, potentially allowing the LEBemitted to use them to be smaller (in a very large function). After thesorting, it also removes locals not used at all.
• SimplifyGlobals - Optimizes globals in various ways, for example,coalescing them, removing mutability from a global never modified, applying aconstant value from an immutable global, etc.
• SimplifyLocals - Key "local.get/set/tee" optimization pass, doing thingslike replacing a set and a get with moving the set’s value to the get (andcreating a tee) where possible. Also createsblock/if/loop return valuesinstead of using a local to pass the value.
• Vacuum - Key "remove silly unneeded code" pass, doing things like removinganif arm that has no contents, a drop of a constant value with no sideeffects, ablock with a single child, etc.

"LTO" in the above means an optimization is Link Time Optimization-like in thatit works across multiple functions, but in a sense Binaryen is always "LTO" asit usually is run on the final linked wasm.

Advanced optimization techniques in the Binaryen optimizer includeSSAification,Flat IR, andStack/Poppy IR.

See theOptimizer Cookbook wiki pagefor more on how to use the optimizer effectively.

Binaryen also contains various passes that do other things than optimizations,likelegalization for JavaScript,Asyncify,etc.

Binaryen uses git submodules (at time of writing just for gtest), so before you build you will have to initialize the submodules:

git submodule initgit submodule update

After that you can build with CMake:

cmake.&& make

A C++17 compiler is required. On macOS, you need to installcmake, for example, viabrew install cmake. Note that you can also useninja as your generator:cmake -G Ninja . && ninja.

To avoid the gtest dependency, you can pass-DBUILD_TESTS=OFF to cmake.

Binaryen.js can be built using Emscripten, which can be installed viathe SDK.

• Building for Node.js:

  emcmake cmake.&& emmake make binaryen_js

• Building for the browser:

  emcmake cmake -DBUILD_FOR_BROWSER=ON.&& emmake make

1. Using the Microsoft Visual Studio Installer, install the "Visual C++ tools for CMake" component.

2. Generate the projects:

   mkdir buildcd build"%VISUAL_STUDIO_ROOT%\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" ..

   Substitute VISUAL_STUDIO_ROOT with the path to your Visual Studioinstallation. In case you are using the Visual Studio Build Tools, the pathwill be "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools".

3. From the Developer Command Prompt, build the desired projects:

   msbuild binaryen.vcxproj

   CMake generates a project named "ALL_BUILD.vcxproj" for conveniently building all the projects.

Builds are distributed by the various toolchains that use Binaryen, likeEmscripten,wasm-pack, etc. There are also official releases on GitHub:

https://github.com/WebAssembly/binaryen/releases

Currently builds of the following platforms are included:

• Linux-x86_64
• Linux-arm64
• MacOS-x86_64
• MacOS-arm64
• Windows-x86_64
• Node.js (experimental): A port ofwasm-opt to JavaScript+WebAssembly.Runnode wasm-opt.js as a drop-in replacement for a native build ofwasm-opt, on any platform that Node.js runs on. Requires Node.js 18+ (forWasm EH and Wasm Threads). (Note that this build may also run in Deno, Bun,or other JavaScript+WebAssembly environments, but is tested only on Node.js.)

Run

bin/wasm-opt [.wasm or .wat file] [options] [passes, see --help] [--help]

The wasm optimizer receives WebAssembly as input, and can run transformationpasses on it, as well as print it (before and/or after the transformations). Forexample, try

bin/wasm-opt test/lit/passes/name-types.wast -all -S -o -

That will output one of the test cases in the test suite. To run atransformation pass on it, try

bin/wasm-opt test/lit/passes/name-types.wast --name-types -all -S -o -

Thename-types pass ensures each type has a name and renames exceptionally long type names. You can seethe change the transformation causes by comparing the output of the two commands.

It's easy to add your own transformation passes to the shell, just add.cppfiles intosrc/passes, and rebuild the shell. For example code, take a look atthename-types pass.

Some more notes:

• Seebin/wasm-opt --help for the full list of options and passes.
• Passing--debug will emit some debugging info. Individual debug channels(defined in the source code via#define DEBUG_TYPE xxx) can be enabled bypassing them as list of comma-separated strings. For example:bin/wasm-opt --debug=binary. These debug channels can also be enabled via theBINARYEN_DEBUG environment variable.

Run

bin/wasm2js [input.wasm file]

This will print out JavaScript to the console.

For example, try

bin/wasm2js test/hello_world.wat

That output contains

functionadd(x,y){x=x|0;y=y|0;returnx+y|0|0;}

as a translation of

 (func$add(; 0 ;) (type$0) (param$xi32) (param$yi32) (resulti32)  (i32.add   (local.get$x)   (local.get$y)  ) )

wasm2js's output is in ES6 module format - basically, it converts a wasmmodule into an ES6 module (to run on older browsers and Node.js versionsyou can use Babel etc. to convert it to ES5). Let's look at a full exampleof calling that hello world wat; first, create the main JS file:

// main.mjsimport{add}from"./hello_world.mjs";console.log('the sum of 1 and 2 is:',add(1,2));

The run this (note that you need a new enough Node.js with ES6 modulesupport):

$ bin/wasm2js test/hello_world.wat -o hello_world.mjs$ node --experimental-modules main.mjsthe sum of 1 and 2 is: 3

Things keep to in mind with wasm2js's output:

• You should run wasm2js with optimizations for release builds, using-Oor another optimization level. That will optimize along the entire pipeline(wasm and JS). It won't do everything a JS minifer would, though, likeminify whitespace, so you should still run a normal JS minifer afterwards.
• It is not possible to match WebAssembly semantics 100% precisely with fastJavaScript code. For example, every load and store may trap, and to makeJavaScript do the same we'd need to add checks everywhere, which would belarge and slow. Instead, wasm2js assumes loads and stores do not trap, thatint/float conversions do not trap, and so forth. There may also be slightdifferences in corner cases of conversions, like non-trapping float to int.

wasm-ctor-eval executes functions, or parts of them, at compile time.After doing so it serializes the runtime state into the wasm, which is liketaking a "snapshot". When the wasm is later loaded and run in a VM, it willcontinue execution from that point, without re-doing the work that was alreadyexecuted.

For example, consider this small program:

(module;; A global variable that begins at 0. (global$global (muti32) (i32.const0)) (import"import""import" (func$import)) (func"main";; Set the global to 1.  (global.set$global   (i32.const1));; Call the imported function. This *cannot* be executed at;; compile time.  (call$import);; We will never get to this point, since we stop at the;; import.  (global.set$global   (i32.const2)) ))

We can evaluate part of it at compile time like this:

wasm-ctor-eval input.wat --ctors=main -S -o -

This tells it that there is a single function that we want to execute ("ctor"is short for "global constructor", a name that comes from code that is executedbefore a program's entry point) and then to print it as text tostdout. Theresult is this:

trying toeval main  ...partial evalling successful, but stopping since could not eval: call import: import.import  ...stopping(module (type$none_=>_none (func)) (import"import""import" (func$import)) (global$global (mut i32) (i32.const 1)) (export"main" (func$0_0)) (func$0_0  (call$import)  (global.set$global   (i32.const 2)  ) ))

The logging shows us managing to eval part ofmain(), but not all of it, asexpected: We can eval the firstglobal.get, but then we stop at the call tothe imported function (because we don't know what that function will be when thewasm is actually run in a VM later). Note how in the output wasm the global'svalue has been updated from 0 to 1, and that the firstglobal.get has beenremoved: the wasm is now in a state that, when we run it in a VM, will seamlesslycontinue to run from the point at whichwasm-ctor-eval stopped.

In this tiny example we just saved a small amount of work. How much work can besaved depends on your program. (It can help to do pure computation up front, andleave calls to imports to as late as possible.)

Note thatwasm-ctor-eval's name is related to global constructor functions,as mentioned earlier, but there is no limitation on what you can execute here.Any export from the wasm can be executed, if its contents are suitable. Forexample, in Emscriptenwasm-ctor-eval is even run onmain() when possible.

wasm-merge combines wasm files together. For example, imagine you have aproject that uses wasm files from multiple toolchains. Then it can be helpful tomerge them all into a single wasm file before shipping, since in a single wasmfile the calls between the modules become just normal calls inside a module,which allows them to be inlined, dead code eliminated, and so forth, potentiallyimproving speed and size.

wasm-merge operates on normal wasm files. It differs fromwasm-ld in thatrespect, aswasm-ld operates on wasmobject files.wasm-merge can helpin multi-toolchain situations where at least one of the toolchains does not usewasm object files.

For example, imagine we have these two wasm files:

;; a.wasm(module  (import"second""bar" (func$second.bar))  (export"main" (func$func))  (func$func    (call$second.bar)  ))

;; b.wasm(module  (import"outside""log" (func$log (parami32)))  (export"bar" (func$func))  (func$func    (call$log      (i32.const42)    )  ))

The filenames on your local drive area.wasm andb.wasm, but for merging /bundling purposes let's say that the first is known as"first" and the secondas"second". That is, we want the first module's import of"second.bar" tocall the function$func in the second module. Here is a wasm-merge command forthat:

wasm-merge a.wasm first b.wasm second -o output.wasm

We give it the first wasm file, then its name, and then the second wasm fileand then its name. The merged output is this:

(module  (import"outside""log" (func$log (parami32)))  (export"main" (func$func))  (export"bar" (func$func_2))  (func$func    (call$func_2)  )  (func$func_2    (call$log      (i32.const42)    )  ))

wasm-merge combined the two files into one, merging their functions, imports,etc., all while fixing up name conflicts and connecting corresponding imports toexports. In particular, note how$func calls$func_2, which is exactly whatwe wanted:$func_2 is the function from the second module (renamed to avoid aname collision).

Note that the wasm output in this example could benefit from additionaloptimization. First, the call to$func_2 can now be easily inlined, so we canrunwasm-opt -O3 to do that for us. Also, we may not need all the imports andexports, for which we can runwasm-metadce.A good workflow could be to runwasm-merge, thenwasm-metadce, then finishwithwasm-opt.

wasm-merge is kind of like a bundler for wasm files, in the sense of a "JSbundler" but for wasm. That is, with the wasm files above, imagine that we hadthis JS code to instantiate and connect them at runtime:

// Compile the first module.varfirst=awaitfetch("a.wasm");first=newWebAssembly.Module(first);// Compile the first module.varsecond=awaitfetch("b.wasm");second=newWebAssembly.Module(second);// Instantiate the second, with a JS import.second=newWebAssembly.Instance(second,{outside:{log:(value)=>{console.log('value:',value);}}});// Instantiate the first, importing from the second.first=newWebAssembly.Instance(first,{second:second.exports});// Call the main function.first.exports.main();

Whatwasm-merge does is basically what that JS does: it hooks up imports toexports, resolving names using the module names you provided. That is, byrunningwasm-merge we are moving the work of connecting the modules fromruntime to compile time. As a result, after runningwasm-merge we need a lotless JS to get the same result:

// Compile the single module.varmerged=awaitfetch("merged.wasm");merged=newWebAssembly.Module(merged);// Instantiate it with a JS import.merged=newWebAssembly.Instance(merged,{outside:{log:(value)=>{console.log('value:',value);}}});// Call the main function.merged.exports.main();

We still need to fetch and compile the merged wasm, and to provide it the JSimport, but the work to connect two wasm modules is not needed any more.

By defaultwasm-merge errors if there are overlapping export names. That is,wasm-merge will automatically handle overlapping function names and so forth,because those are not externally visible (the code still behaves the same), butif we renamed exports then the outside would need to be modified to expect thenew export names, and so we error instead on such name conflicts.

If you do want exports to be renamed, runwasm-merge with--rename-export-conflicts. Later exports will have a suffix appended to themto ensure they do not overlap with previous exports. The suffixes aredeterministic, so once you see what they are you can call them from the outside.

Another option is to use--skip-export-conflicts which will simply skip laterexports that have conflicting names. For example, this can be useful in thecase where the first module is the only one that interacts with the outside andthe later modules just interact with the first module.

wasm-merge uses the multi-memory and multi-table features. That is, ifmultiple input modules each have a memory then the output wasm will have severalmemories, and will depend on the multi-memory feature, which means that olderwasm VMs might not be able to run the wasm. (As a workaround for such older VMsyou can runwasm-opt --multi-memory-lowering to lower multiple memories into asingle one.)

./check.py

(orpython check.py) will runwasm-shell,wasm-opt, etc. on the testcases intest/, and verify their outputs.

Thecheck.py script supports some options:

./check.py [--interpreter=/path/to/interpreter] [TEST1] [TEST2]..

• If an interpreter is provided, we run the output through it, checking forparse errors.
• If tests are provided, we run exactly those. If none are provided, we runthem all. To see what tests are available, run./check.py --list-suites.
• Some tests requireemcc ornodejs in the path. They will not run if thetool cannot be found, and you'll see a warning.
• We have tests from upstream intests/spec, in git submodules. Running./check.py should update those.

Note that we are trying to gradually port the legacy wasm-opt tests to uselitandfilecheck as we modify them. Forpasses tests that output wast, thiscan be done automatically withscripts/port_passes_tests_to_lit.py and fornon-passes tests that output wast, see#4779 for an example of how to do asimple manual port.

For lit tests the test expectations (the CHECK lines) can often be automaticallyupdated as changes are made to binaryen. Seescripts/update_lit_checks.py.

Non-lit tests can also be automatically updated in most cases. Seescripts/auto_update_tests.py.

./third_party/setup.py [mozjs|v8|wabt|all]

(orpython third_party/setup.py) installs required dependencies like the SpiderMonkey JS shell, the V8 JS shelland WABT inthird_party/. Other scripts automatically pick these up when installed.

Runpip3 install -r requirements-dev.txt to get the requirements for thelittests. Note that you need to have the locationpip installs to in your$PATH(on linux,~/.local/bin).

./scripts/fuzz_opt.py [--binaryen-bin=build/bin]

(orpython scripts/fuzz_opt.py) will run various fuzzing modes on random inputs with random passes until it findsa possible bug. Seethe wiki page for all the details.

• Interned strings for names: It's very convenient to have names on nodes,instead of just numeric indices etc. To avoid most of the performancedifference between strings and numeric indices, all strings are interned,which means there is a single copy of each string in memory, stringcomparisons are just a pointer comparison, etc.
• Allocate in arenas: Based on experience with otheroptimizing/transformating toolchains, it's not worth the overhead tocarefully track memory of individual nodes. Instead, we allocate all elementsof a module in an arena, and the entire arena can be freed when the module isno longer needed.

Binaryen can read and write source maps (see the-ism and-osm flags towasm-opt). It can also read and read source map annotations in the textformat, that is,

;;@ src.cpp:100:33(i32.const42)

That 42 constant is annotated as appearing in a file calledsrc.cpp at line100 and column33. Source maps and text format annotations areinterchangeable, that is, they both lead to the same IR representation, so youcan start with an annotated wat and have Binaryen write that to a binary + asource map file, or read a binary + source map file and print text which willcontain those annotations.

The IR representation of source map info is simple: in each function we have amap of expressions to their locations. Optimization passes should update themap as relevant. Often this "just works" because the optimizer tries to reusenodes when possible, so they keep the same debug info.

The text format annotations support a shorthand in which repeated annotationsare not necessary. For example, children are tagged with the debug info of theparent, if they have no annotation of their own:

;;@ src.cpp:100:33(i32.add  (i32.const41);; This receives an annotation of src.cpp:100:33;;@ src.cpp:111:44  (i32.const1))

The first const will have debug info identical to the parent, because it hasnone specified, and generally such nesting indicates a "bundle" of instructionsthat all implement the same source code.

Note that text printing will not emit such repeated annotations, which can beconfusing. To print out all the annotations, setBINARYEN_PRINT_FULL=1 in theenvironment. That will print this for the aboveadd:

[i32];;@ src.cpp:100:33(i32.add [i32];;@ src.cpp:100:33 (i32.const41) [i32];;@ src.cpp:111:44 (i32.const1))

(full print mode also adds a[type] for each expression, right before thedebug location).

The debug information is also propagated from an expression to itsnext sibling:

;;@ src.cpp:100:33(local.set$x (i32.const0))(local.set$y;; This receives an annotation of src.cpp:100:33 (i32.const0))

You can prevent the propagation of debug info by explicitly mentioningthat an expression has not debug info using the annotation;;@ withnothing else:

;;@ src.cpp:100:33(local.set$x;;@ (i32.const0);; This does not receive any annotation);;@(local.set$y;; This does not receive any annotation (i32.const7))

This stops the propagatation to children and siblings as well. So,expression(i32.const 7) does not have any debug info either.

There is no shorthand in the binary format. That is, roundtripping (writing andreading) through a binary + source map should not change which expressions havedebug info on them or the contents of that info.

Thesource maps format defines a mappingusingsegments, that is, if a segment starts at binary offset 10 then itapplies to all instructions at that offset and until another segment begins (orthe end of the input is reached). Binaryen's IR represents a mapping fromexpressions to locations, as mentioned, so we need to map to and from thesegment-based format when writing and reading source maps.

That is mostly straightforward, but one thing we need to do is to handle thelack of debug info in between things that have it. If we haveA B C whereBlacks debug info, then just emitting a segment forA andC would leadA'ssegment to also coverB, since in source maps segments do not have a size -rather they end when a new segment begins. To avoidB getting smeared in thismanner, we emit a source maps entry toB of size 1, which just marks thebinary offset it has, and without the later 3 fields of the source file, linenumber, and column. (This appears to be the intent of the source maps spec, andworks in browsers and tools.)

Binaryen also has optional support for DWARF. This primarily just tracks thelocations of expressions and rewrites the DWARF's locations accordingly; it doesnot handle things like re-indexing of locals, and so passes that might breakDWARF are disabled by default. As a result, this mode is not suitable for afully optimized release build, but it can be useful for local debugging.

• Why the weird name for the project?

Binaryen's name was inspired byEmscripten's: Emscripten's namesuggestsit converts something into ascript - specificallyJavaScript - andBinaryen's suggests it converts something into abinary - specificallyWebAssembly. Binaryen began as Emscripten's WebAssembly generation andoptimization tool, so the name fit as it moved Emscripten from something thatemitted the text-based format JavaScript (as it did from its early days) to thebinary format WebAssembly (which it has done since WebAssembly launched).

"Binaryen" is pronouncedin the same manneras"Targaryen".

• Does it compile under Windows and/or Visual Studio?

Yes, it does. Here's a step-by-steptutorial on how to compile itunderWindows 10 x64 with withCMake andVisual Studio 2015.However, Visual Studio 2017 may now be required. Help would be appreciated onWindows and OS X as most of the core devs are on Linux.