Extism Go SDK

This repo houses the Go SDK for integrating with theExtism runtime. Install this library into your host Go applications to run Extism plugins.

Join theExtism Discord and chat with us!

Installation

Install viago get:

go get github.com/extism/go-sdk

Reference Docs

You can find the reference docs athttps://pkg.go.dev/github.com/extism/go-sdk.

Getting Started

This guide should walk you through some of the concepts in Extism and this Go library.

Creating A Plug-in

The primary concept in Extism is theplug-in. You can think of a plug-in as a code module stored in a.wasm file.

Plug-in code can come from a file on disk, object storage or any number of places. Since you may not have one handy let's load a demo plug-in from the web. Let'sstart by creating a main func and loading an Extism Plug-in:

package mainimport ("context""fmt""github.com/extism/go-sdk""os")func main() {manifest := extism.Manifest{Wasm: []extism.Wasm{extism.WasmUrl{Url: "https://github.com/extism/plugins/releases/latest/download/count_vowels.wasm",},},}ctx := context.Background()config := extism.PluginConfig{}plugin, err := extism.NewPlugin(ctx, manifest, config, []extism.HostFunction{})if err != nil {fmt.Printf("Failed to initialize plugin: %v\n", err)os.Exit(1)}}

Note: Seethe Manifest docs as it has a rich schema and a lot of options.

Calling A Plug-in's Exports

This plug-in was written in Rust and it does one thing, it counts vowels in a string. As such, it exposes one "export" function:count_vowels. We can call exports usingextism.Plugin.Call.Let's add that code to our main func:

func main() {    // ...data := []byte("Hello, World!")exit, out, err := plugin.Call("count_vowels", data)if err != nil {fmt.Println(err)os.Exit(int(exit))}response := string(out)fmt.Println(response)    // => {"count": 3, "total": 3, "vowels": "aeiouAEIOU"}}

Running this should print out the JSON vowel count report:

$ go run main.go# => {"count":3,"total":3,"vowels":"aeiouAEIOU"}

All exports have a simple interface of optional bytes in, and optional bytes out. This plug-in happens to take a string and return a JSON encoded string with a report of results.

Note: If you want to pass a customcontext.Context when calling a plugin function, you can use theextism.Plugin.CallWithContext method instead.

Plug-in State

Plug-ins may be stateful or stateless. Plug-ins can maintain state between calls by the use of variables. Our count vowels plug-in remembers the total number of vowels it's ever counted in the "total" key in the result. You can see this by making subsequent calls to the export:

func main () {    // ...    exit, out, err := plugin.Call("count_vowels", []byte("Hello, World!"))    if err != nil {        fmt.Println(err)        os.Exit(int(exit))    }    fmt.Println(string(out))    // => {"count": 3, "total": 6, "vowels": "aeiouAEIOU"}    exit, out, err = plugin.Call("count_vowels", []byte("Hello, World!"))    if err != nil {        fmt.Println(err)        os.Exit(int(exit))    }    fmt.Println(string(out))    // => {"count": 3, "total": 9, "vowels": "aeiouAEIOU"}}

These variables will persist until this plug-in is freed or you initialize a new one.

Configuration

Plug-ins may optionally take a configuration object. This is a static way to configure the plug-in. Our count-vowels plugin takes an optional configuration to change out which characters are considered vowels. Example:

func main() {    manifest := extism.Manifest{        Wasm: []extism.Wasm{            extism.WasmUrl{                Url: "https://github.com/extism/plugins/releases/latest/download/count_vowels.wasm",            },        },        Config: map[string]string{            "vowels": "aeiouyAEIOUY",        },    }    ctx := context.Background()    config := extism.PluginConfig{}    plugin, err := extism.NewPlugin(ctx, manifest, config, []extism.HostFunction{})    if err != nil {        fmt.Printf("Failed to initialize plugin: %v\n", err)        os.Exit(1)    }    exit, out, err := plugin.Call("count_vowels", []byte("Yellow, World!"))    if err != nil {        fmt.Println(err)        os.Exit(int(exit))    }    fmt.Println(string(out))    // => {"count": 4, "total": 4, "vowels": "aeiouAEIOUY"}}

Host Functions

Let's extend our count-vowels example a little bit: Instead of storing thetotal in an ephemeral plug-in var, let's store it in a persistent key-value store!

Wasm can't use our KV store on it's own. This is whereHost Functions come in.

Host functions allow us to grant new capabilities to our plug-ins from our application. They are simply some Go functions you write which can be passed down and invoked from any language inside the plug-in.

Let's load the manifest like usual but load up thiscount_vowels_kvstore plug-in:

manifest := extism.Manifest{    Wasm: []extism.Wasm{        extism.WasmUrl{            Url: "https://github.com/extism/plugins/releases/latest/download/count_vowels_kvstore.wasm",        },    },}

Note: The source code for this ishere and is written in rust, but it could be written in any of our PDK languages.

Unlike our previous plug-in, this plug-in expects you to provide host functions that satisfy our its import interface for a KV store.

We want to expose two functions to our plugin,kv_write(key string, value []bytes) which writes a bytes value to a key andkv_read(key string) []byte which reads the bytes at the givenkey.

// pretend this is Redis or something :)kvStore := make(map[string][]byte)kvRead := extism.NewHostFunctionWithStack(    "kv_read",    func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {        key, err := p.ReadString(stack[0])        if err != nil {            panic(err)        }        value, success := kvStore[key]        if !success {            value = []byte{0, 0, 0, 0}        }        stack[0], err = p.WriteBytes(value)    },    []ValueType{ValueTypePTR},    []ValueType{ValueTypePTR},)kvWrite := extism.NewHostFunctionWithStack(    "kv_write",    func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) {        key, err := p.ReadString(stack[0])        if err != nil {            panic(err)        }        value, err := p.ReadBytes(stack[1])        if err != nil {            panic(err)        }        kvStore[key] = value    },    []ValueType{ValueTypePTR, ValueTypePTR},    []ValueType{},)

Note: In order to write host functions you should get familiar with the methods on theextism.CurrentPlugin type. Thep parameter is an instance of this type.

We need to pass these imports to the plug-in to create them. All imports of a plug-in must be satisfied for it to be initialized:

plugin, err := extism.NewPlugin(ctx, manifest, config, []extism.HostFunction{kvRead, kvWrite});

Now we can invoke the event:

exit, out, err := plugin.Call("count_vowels", []byte("Hello, World!"))// => Read from key=count-vowels"// => Writing value=3 from key=count-vowels"// => {"count": 3, "total": 3, "vowels": "aeiouAEIOU"}exit, out, err = plugin.Call("count_vowels", []byte("Hello, World!"))// => Read from key=count-vowels"// => Writing value=6 from key=count-vowels"// => {"count": 3, "total": 6, "vowels": "aeiouAEIOU"}

Enabling Compilation Cache

While Wazero (the underlying Wasm runtime) is very fast in initializing modules, you can make subsequent initializations even faster by enabling the compilation cache:

ctx := context.Background()cache := wazero.NewCompilationCache()defer cache.Close(ctx)manifest := Manifest{Wasm: []Wasm{WasmFile{Path: "wasm/noop.wasm"}}}config := PluginConfig{    EnableWasi:    true,    ModuleConfig:  wazero.NewModuleConfig(),    RuntimeConfig: wazero.NewRuntimeConfig().WithCompilationCache(cache),}_, err := NewPlugin(ctx, manifest, config, []HostFunction{})

Integrate with Dylibso Observe SDK

Dylibso providesobservability SDKs for WebAssembly (Wasm), enabling continuous monitoring of WebAssembly code as it executes within a runtime. It provides developers with the tools necessary to capture and emit telemetry data from Wasm code, including function execution and memory allocation traces, logs, and metrics.

While Observe SDK has adapters for many popular observability platforms, it also ships with an stdout adapter:

ctx := context.Background()adapter := stdout.NewStdoutAdapter()adapter.Start(ctx)manifest := manifest("nested.c.instr.wasm")config := PluginConfig{    ModuleConfig:   wazero.NewModuleConfig().WithSysWalltime(),    EnableWasi:     true,    ObserveAdapter: adapter.AdapterBase,}plugin, err := NewPlugin(ctx, manifest, config, []HostFunction{})if err != nil {    panic(err)}meta := map[string]string{    "http.url":         "https://example.com/my-endpoint",    "http.status_code": "200",    "http.client_ip":   "192.168.1.0",}plugin.TraceCtx.Metadata(meta)_, _, _ = plugin.Call("_start", []byte("hello world"))plugin.Close()

Enable filesystem access

WASM plugins can read/write files outside the runtime. To do this we addAllowedPaths mapping of "HOST:PLUGIN" to theextism.Manifest of our plugin.

package mainimport ("context""fmt""os"extism "github.com/extism/go-sdk")func main() {manifest := extism.Manifest{AllowedPaths: map[string]string{// Here we specifify a host directory data to be linked// to the /mnt directory inside the wasm runtime"data": "/mnt",},Wasm: []extism.Wasm{extism.WasmFile{Path: "fs_plugin.wasm",},},}ctx := context.Background()config := extism.PluginConfig{EnableWasi: true,}plugin, err := extism.NewPlugin(ctx, manifest, config, []extism.HostFunction{})if err != nil {fmt.Printf("Failed to initialize plugin: %v\n", err)os.Exit(1)}data := []byte("Hello world, this is written from within our wasm plugin.")exit, _, err := plugin.Call("write_file", data)if err != nil {fmt.Println(err)os.Exit(int(exit))}}

Note: In order for filesystem APIs to work the plugin needs to be compiled with WASI target. Source code for the plugin can be foundhere and is written in Go, but it could be written in any of our PDK languages.

Build example plugins

Since ourexample plugins are also written in Go, for compiling them we useTinyGo:

cd plugins/configtinygo build -target wasi -o ../wasm/config.wasm main.go

Constants¶

const (// ValueTypeI32 is a 32-bit integer.ValueTypeI32 =api.ValueTypeI32// ValueTypeI64 is a 64-bit integer.ValueTypeI64 =api.ValueTypeI64// ValueTypeF32 is a 32-bit floating point number.ValueTypeF32 =api.ValueTypeF32// ValueTypeF64 is a 64-bit floating point number.ValueTypeF64 =api.ValueTypeF64// ValueTypePTR represents a pointer to an Extism memory block. Alias for ValueTypeI64ValueTypePTR =ValueTypeI64)

const (None runtimeType =iotaHaskellWasi)

Variables¶

Functions¶

func DecodeF32(inputuint64)float32

func DecodeF64(inputuint64)float64

func DecodeI32(inputuint64)int32

func DecodeU32(inputuint64)uint32

func EncodeF32(inputfloat32)uint64

func EncodeF64(inputfloat64)uint64

func EncodeI32(inputint32)uint64

func EncodeI64(inputint64)uint64

func EncodeU32(inputuint32)uint64

func RuntimeVersion()string

func SetLogLevel(levelLogLevel)

Types¶

type CompiledPlugin struct {// contains filtered or unexported fields}

func NewCompiledPlugin(ctxcontext.Context,manifestManifest,configPluginConfig,funcs []HostFunction,) (*CompiledPlugin,error)

func (p *CompiledPlugin) Close(ctxcontext.Context)error

func (p *CompiledPlugin) Instance(ctxcontext.Context, configPluginInstanceConfig) (*Plugin,error)

type CurrentPlugin struct {// contains filtered or unexported fields}

func (p *CurrentPlugin) Alloc(nuint64) (uint64,error)

func (p *CurrentPlugin) AllocWithContext(ctxcontext.Context, nuint64) (uint64,error)

func (p *CurrentPlugin) Free(offsetuint64)error

func (p *CurrentPlugin) FreeWithContext(ctxcontext.Context, offsetuint64)error

func (p *CurrentPlugin) Length(offsuint64) (uint64,error)

func (p *CurrentPlugin) LengthWithContext(ctxcontext.Context, offsuint64) (uint64,error)

func (p *CurrentPlugin) Log(levelLogLevel, messagestring)

func (p *CurrentPlugin) Logf(levelLogLevel, formatstring, args ...any)

func (p *CurrentPlugin) Memory()api.Memory

func (p *CurrentPlugin) ReadBytes(offsetuint64) ([]byte,error)

func (p *CurrentPlugin) ReadString(offsetuint64) (string,error)

func (p *CurrentPlugin) WriteBytes(b []byte) (uint64,error)

func (p *CurrentPlugin) WriteString(sstring) (uint64,error)

type FunctionDefinition struct {// contains filtered or unexported fields}

func (f *FunctionDefinition) Name()string

type HostFunction struct {NamestringNamespacestringParams    []api.ValueTypeReturns   []api.ValueType// contains filtered or unexported fields}

func NewHostFunctionWithStack(namestring,callbackHostFunctionStackCallback,params []ValueType,returnTypes []ValueType)HostFunction

 mult := NewHostFunctionWithStack("mult",func(ctx context.Context, plugin *CurrentPlugin, stack []uint64) {a := DecodeI32(stack[0])b := DecodeI32(stack[1])stack[0] = EncodeI32(a * b)},[]ValueType{ValueTypeI64, ValueTypeI64},ValueTypeI64 )

func (f *HostFunction) SetNamespace(namespacestring)

type HostFunctionStackCallback func(ctxcontext.Context, p *CurrentPlugin, stack []uint64)

// read parameters in index orderargv, argvBuf := DecodeU32(inputs[0]), DecodeU32(inputs[1])// write results back to the stack in index orderstack[0] = EncodeU32(ErrnoSuccess)

type HttpRequest struct {UrlstringHeaders map[string]stringMethodstring}

type InputOffsetKeystring

type LogLevelint32

const (LogLevelTrace LogLevelLogLevelDebugLogLevelInfoLogLevelWarnLogLevelErrorLogLevelOffLogLevel =math.MaxInt32)

func (lLogLevel) ExtismCompat()int32

func (lLogLevel) String()string

type Manifest struct {Wasm         []Wasm            `json:"wasm"`Memory       *ManifestMemory   `json:"memory,omitempty"`Config       map[string]string `json:"config,omitempty"`AllowedHosts []string          `json:"allowed_hosts,omitempty"`AllowedPaths map[string]string `json:"allowed_paths,omitempty"`Timeoutuint64            `json:"timeout_ms,omitempty"`}

func (m *Manifest) UnmarshalJSON(data []byte)error

type ManifestMemory struct {MaxPagesuint32 `json:"max_pages,omitempty"`MaxHttpResponseBytesint64  `json:"max_http_response_bytes,omitempty"`MaxVarBytesint64  `json:"max_var_bytes,omitempty"`}

type Module struct {// contains filtered or unexported fields}

func (m *Module) ExportedFunctions() map[string]FunctionDefinition

type Plugin struct {Timeouttime.DurationConfig               map[string]stringVar                  map[string][]byteAllowedHosts         []stringAllowedPaths         map[string]stringLastStatusCodeintLastResponseHeaders  map[string]stringMaxHttpResponseBytesint64MaxVarBytesint64Adapter *observe.AdapterBase// contains filtered or unexported fields}

func NewPlugin(ctxcontext.Context,manifestManifest,configPluginConfig,functions []HostFunction,) (*Plugin,error)

func (p *Plugin) Call(namestring, data []byte) (uint32, []byte,error)

func (p *Plugin) CallWithContext(ctxcontext.Context, namestring, data []byte) (uint32, []byte,error)

func (p *Plugin) Close(ctxcontext.Context)error

func (p *Plugin) CloseWithContext(ctxcontext.Context)error

func (p *Plugin) FunctionExists(namestring)bool

func (p *Plugin) GetError()string

func (p *Plugin) GetErrorWithContext(ctxcontext.Context)string

func (p *Plugin) GetOutput() ([]byte,error)

func (p *Plugin) GetOutputWithContext(ctxcontext.Context) ([]byte,error)

func (p *Plugin) Log(levelLogLevel, messagestring)

func (p *Plugin) Logf(levelLogLevel, formatstring, args ...any)

func (p *Plugin) Memory()api.Memory

func (p *Plugin) Module() *Module

func (p *Plugin) SetInput(data []byte) (uint64,error)

func (p *Plugin) SetInputWithContext(ctxcontext.Context, data []byte) (uint64,error)

func (p *Plugin) SetLogger(logger func(LogLevel,string))

type PluginConfig struct {RuntimeConfigwazero.RuntimeConfigEnableWasiboolObserveAdapter            *observe.AdapterBaseObserveOptions            *observe.OptionsEnableHttpResponseHeadersbool// ModuleConfig is only used when a plugins are built using the NewPlugin// function. In this function, the plugin is both compiled, and an instance// of the plugin is instantiated, and the ModuleConfig is passed to the// instance.//// When plugins are built using NewCompiledPlugin, the ModuleConfig has no// effect because the instance is not created. Instead, the ModuleConfig is// passed directly in calls to the CompiledPlugin.Instance method.//// NOTE: Module name and start functions are ignored as they are overridden by Extism, also if Manifest contains// non-empty AllowedPaths, then FS is also ignored. If EXTISM_ENABLE_WASI_OUTPUT is set, then stdout and stderr are// set to os.Stdout and os.Stderr respectively (ignoring user defined module config).ModuleConfigwazero.ModuleConfig}

type PluginCtxKeystring

type PluginInstanceConfig struct {// ModuleConfig allows the user to specify custom module configuration.//// NOTE: Module name and start functions are ignored as they are overridden by Extism, also if Manifest contains// non-empty AllowedPaths, then FS is also ignored. If EXTISM_ENABLE_WASI_OUTPUT is set, then stdout and stderr are// set to os.Stdout and os.Stderr respectively (ignoring user defined module config).ModuleConfigwazero.ModuleConfig}

type Runtime struct {Wazerowazero.RuntimeExtismapi.ModuleEnvapi.Module}

type ValueType =byte

type Wasm interface {ToWasmData(ctxcontext.Context) (WasmData,error)}

type WasmData struct {Data []byte `json:"data"`Hashstring `json:"hash,omitempty"`Namestring `json:"name,omitempty"`}

func (dWasmData) ToWasmData(ctxcontext.Context) (WasmData,error)

type WasmFile struct {Pathstring `json:"path"`Hashstring `json:"hash,omitempty"`Namestring `json:"name,omitempty"`}

func (fWasmFile) ToWasmData(ctxcontext.Context) (WasmData,error)

type WasmUrl struct {Urlstring            `json:"url"`Hashstring            `json:"hash,omitempty"`Headers map[string]string `json:"headers,omitempty"`Namestring            `json:"name,omitempty"`Methodstring            `json:"method,omitempty"`}

func (uWasmUrl) ToWasmData(ctxcontext.Context) (WasmData,error)