Repository files navigation

go-plugin is a Go (golang) plugin system over RPC. It is the plugin systemthat has been in use by HashiCorp tooling for over 4 years. While initiallycreated forPacker, it is additionally in use byTerraform,Nomad,Vault,Boundary,andWaypoint.

While the plugin system is over RPC, it is currently only designed to workover a local [reliable] network. Plugins over a real network are not supportedand will lead to unexpected behavior.

This plugin system has been used on millions of machines across many differentprojects and has proven to be battle hardened and ready for production use.

The HashiCorp plugin system supports a number of features:

Plugins are Go interface implementations. This makes writing and consumingplugins feel very natural. To a plugin author: you just implement aninterface as if it were going to run in the same process. For a plugin user:you just use and call functions on an interface as if it were in the sameprocess. This plugin system handles the communication in between.

Cross-language support. Plugins can be written (and consumed) byalmost every major language. This library supports serving plugins viagRPC. gRPC-based plugins enable plugins to be writtenin any language.

Complex arguments and return values are supported. This libraryprovides APIs for handling complex arguments and return values suchas interfaces,io.Reader/Writer, etc. We do this by giving you a library(MuxBroker) for creating new connections between the client/server toserve additional interfaces or transfer raw data.

Bidirectional communication. Because the plugin system supportscomplex arguments, the host process can send it interface implementationsand the plugin can call back into the host process.

Built-in Logging. Any plugins that use thelog standard librarywill have log data automatically sent to the host process. The hostprocess will mirror this output prefixed with the path to the pluginbinary. This makes debugging with plugins simple. If the host systemuseshclog then the log datawill be structured. If the plugin also uses hclog, logs from the pluginwill be sent to the host hclog and be structured.

Protocol Versioning. A very basic "protocol version" is supported thatcan be incremented to invalidate any previous plugins. This is useful wheninterface signatures are changing, protocol level changes are necessary,etc. When a protocol version is incompatible, a human friendly errormessage is shown to the end user.

Stdout/Stderr Syncing. While plugins are subprocesses, they can continueto use stdout/stderr as usual and the output will get mirrored back tothe host process. The host process can control whatio.Writer thesestreams go to to prevent this from happening.

TTY Preservation. Plugin subprocesses are connected to the identicalstdin file descriptor as the host process, allowing software that requiresa TTY to work. For example, a plugin can executessh and even though thereare multiple subprocesses and RPC happening, it will look and act perfectlyto the end user.

Host upgrade while a plugin is running. Plugins can be "reattached"so that the host process can be upgraded while the plugin is still running.This requires the host/plugin to know this is possible and daemonizeproperly.NewClient takes aReattachConfig to determine if and how toreattach.

Cryptographically Secure Plugins. Plugins can be verified with an expectedchecksum and RPC communications can be configured to use TLS. The host processmust be properly secured to protect this configuration.

The HashiCorp plugin system works by launching subprocesses and communicatingover RPC (using standardnet/rpc orgRPC). A singleconnection is made between any plugin and the host process. For net/rpc-basedplugins, we use aconnection multiplexinglibrary to multiplex any other connections on top. For gRPC-based plugins,the HTTP2 protocol handles multiplexing.

This architecture has a number of benefits:

• Plugins can't crash your host process: A panic in a plugin doesn'tpanic the plugin user.

• Plugins are very easy to write: just write a Go application andgo build.Or use any other language to write a gRPC server with a tiny amount ofboilerplate to support go-plugin.

• Plugins are very easy to install: just put the binary in a location wherethe host will find it (depends on the host but this library also provideshelpers), and the plugin host handles the rest.

• Plugins can be relatively secure: The plugin only has access to theinterfaces and args given to it, not to the entire memory space of theprocess. Additionally, go-plugin can communicate with the plugin overTLS.

To use the plugin system, you must take the following steps. These arehigh-level steps that must be done. Examples are available in theexamples/ directory.

1. Choose the interface(s) you want to expose for plugins.

2. For each interface, implement an implementation of that interfacethat communicates over anet/rpc connection or over agRPC connection or both. You'll have to implementboth a client and server implementation.

3. Create aPlugin implementation that knows how to create the RPCclient/server for a given plugin type.

4. Plugin authors callplugin.Serve to serve a plugin from themain function.

5. Plugin users useplugin.Client to launch a subprocess and requestan interface implementation over RPC.

That's it! In practice, step 2 is the most tedious and time consuming step.Even so, it isn't very difficult and you can see examples in theexamples/directory as well as throughout our various open source projects.

For complete API documentation, seeGoDoc.

Our plugin system is constantly evolving. As we use the plugin system fornew projects or for new features in existing projects, we constantly findimprovements we can make.

At this point in time, the roadmap for the plugin system is:

Semantic Versioning. Plugins will be able to implement a semantic version.This plugin system will give host processes a system for constrainingversions. This is in addition to the protocol versioning already presentwhich is more for larger underlying changes.

When we started using plugins (late 2012, early 2013), plugins over RPCwere the only option since Go didn't support dynamic library loading. Today,Go supports theplugin standard library witha number of limitations. Since 2012, our plugin system has stabilizedfrom tens of millions of users using it, and has many benefits we've come tovalue greatly.

For example, we use this plugin system inVault where dynamic library loading isnot acceptable for security reasons. That is an extremeexample, but we believe our library system has more upsides than downsidesover dynamic library loading and since we've had it built and tested for years,we'll continue to use it.

Shared libraries have one major advantage over our system which is muchhigher performance. In real world scenarios across our various tools,we've never required any more performance out of our plugin system and ithas seen very high throughput, so this isn't a concern for us at the moment.