Repository files navigation

Virtualgo (orvg for short) is a tool which provides workspace baseddevelopment for Go. Its main feature set that makes it better than othersolutions is as follows:

1. Extreme ease of use
2. No interference with other go tools
3. Version pinning for imports
4. Version pinning for executables, such as linters (e.g.errcheck) and codegen tools (e.g.protoc-gen-go)
5. Importing a dependency that's locally checked out outside of the workspace(also called multi project workflow)
6. Optional full isolation for imports, see the section onimportmodes for details.

Virtualgo doesn't do dependency resolution or version pinning itself, becausethis is a hard problem that's already being solved by other tools. Its approachis to build on top of these tools, such asdep, to provide the features features listedabove.For people coming from Pythonvg is very similar tovirtualenv, withdepbeing respective topip. The main difference is thatvg is much easier touse thanvirtualenv, because there's almost no mental overhead in usingvg.

The Go community is now using Go Modules to handle dependencies. This project is now mostly unmaintained. Please read more about thishere.

Below is an example showing some basic usage ofvg. See further down andvg helpfor more information and examples.

$cd$GOPATH/src/github.com/GetStream/example$ vg init# initial creation of workspace# Now all commands will be executed from within the example workspace(example) $ go get github.com/pkg/errors# package only present in workspace(example) $ vg ensure# installs the dependencies of the example project using dep(example) $ vg deactivate$cd~$cd$GOPATH/src/github.com/GetStream/example(example) $# The workspace is now activated automatically after cd-ing to the project directory

The obvious question is: Why should you usevg? What advantages does itbring over what you're using now? This obviously depends on what you're usingnow:

1. You can pin versions of executable dependencies, such as linting and codegeneration tools.
2. No more issues withgo test ./... running tests in thevendor directorywhen usinggo 1.8 and below.
3. You can easily use a dependency from your globalGOPATH inside yourworkspace, without running into confusing import errors.
4. It has optionalfull isolation. If enabled there'sno accidental fallbacks to regularGOPATH causing confusion about whatversion of a package you're using.
5. When using full isolation, tools such as IDEs can spend much less time onindexing. This is simply because they don't have to index the packagesoutside the workspace.
6. You don't have problems when using plugins:https://github.com/akutz/gpd

1. Automatic activation of aGOPATH when youcd into a directory.
2. Integration with version management tools such asdep andglide allow forreproducible builds.
3. Useful commands to manage installed packages. For instance for uninstallinga package or installing a local package from anotherGOPATH.

First install the package:

go get -u github.com/GetStream/vg

Although not required, it is recommended to installbindfs as well. This gives the best experience whenusingfull isolation and when usingvg localInstall. If you do this, DON'T remove things manually from~/.virtualgo. Only usevg destroy/vg uninstall, otherwise you can verywell lose data.

# OSXbrew install bindfs# Ubuntuapt install bindfs# Arch Linuxpacaur -S bindfs# or yaourt or whatever tool you use for AUR

You can run the following command to configure all supported shellsautomatically:

vg setup

After this you have to reload (source) your shell configuration file:

source~/.bashrc# for bashsource~/.zshrc# for zshsource~/.config/fish/config.fish# for fish

You can also edit your shell configuration file manually. Afterwards you stillhave tosource the file like explained above.

For bash put this in your~/.bashrc file:

command -v vg>/dev/null2>&1&&eval"$(vgeval --shell bash)"

Or for zsh, put his in your~/.zshrc file:

command -v vg>/dev/null2>&1&&eval"$(vgeval --shell zsh)"

Or for fish, put this in your~/.config/fish/config.fish file:

command-v vg>/dev/null2>&1;and vgeval--shell fish| source

The following commands are the main commands to usevg:

# The first command to use is the one to create and activate a workspace named# after the current direcory$cd$GOPATH/src/github.com/GetStream/example$ vg init(example) $# This command also links the current directory to the created workspace. This# way the next time you cd to this directory the workspace will be activated# automatically.# (See below in the README on how to use the workspace from an IDE)# All go commands in this shell are now executed from within your workspace. The# following will install the most recent version of the cobra command and# library inside the workspace(example) $ go get -u github.com/spf13/cobra/cobra(example) $ cobraCobra is a CLI libraryfor Go that empowers applications.......# It's also possible to only activate a workspace and not link it to the# current directory. If the workspace doesn't exist it will also be# created on the fly. Activating a new workspace automatically deactivates# a previous one:(example) $ vg activate example2(example2) $ cobrabash: cobra:command not found# To deactivate the workspace simply run:(example2) $ vg deactivate$ vg activate(example) $# When a workspace is active, a go compilation will try to import packages# installed from the workspace first. In some cases you might want to use the# version of a package that is installed in your global GOPATH though. For# instance when you are fixing a bug in a dependency and want to test the fix.# In these cases you can easily install a package from your global GOPATH# into the workspace:(example) $ vg localInstall github.com/GetStream/utils# You can even install a package from a specific path:(example) $ vg localInstall github.com/GetStream/utils~/weird/path/utils# You can also uninstall a package from your workspace again(example) $ vg uninstall github.com/spf13/cobra# NOTE: At the moment this only removes the sources and static libs in pkg/, not# executables. So the cobra command is still available.# See the following sections for integration with dependency management tools.# And for a full overview of all commands just run:(example) $ vghelp# For detailed help of a specific command run:(example) $ vghelp<command>

vg integrates well withdep (https://github.com/golang/dep):

# Install the dependencies from Gopkg.lock into your workspace instead of the# vendor directoryvg ensure# Pass options to `dep ensure`vg ensure -- -v -update

It also extendsdep with a way to install executable dependencies. Thevgrepo itself uses it to install thego-bindata andcobra command. It doesthis by adding the following inGopkg.toml:

required = ['github.com/jteeuwen/go-bindata/go-bindata','github.com/spf13/cobra/cobra']

Runningvg ensure after adding this will install thego-bindata andcobracommand in theGOBIN of the current workspace.

As you just sawvg reuses therequiredlist fromdep.However, if you don't want to installall packages in therequired list you can achieve that by putting thefollowing inGopkg.toml:

[metadata]install-required =false

You can also specify which packages to install without therequired list:

[metadata]install = ['github.com/jteeuwen/go-bindata/go-bindata','github.com/golang/mock/...',# supports pkg/... syntax]

Even thoughdep is the main tool that virtualgo integrates with. It's also possibleto use other dependency management tools instead, as long as they create avendor directory. Installing executable dependencies is not supported though(PRs for this are welcome).

To usevg withglide works like this:

# Install dependencies into vendor with glideglide install# Move these dependencies into the workspacevg moveVendor

A workspace can be set up in two different import modes, global fallback or fullisolation.The import mode of a workspace determines how imports from code behave and it ischosen when the workspace is created.

In global fallback mode, packages are imported from the originalGOPATH whenthey are not found in the workspace.This is the default import mode for newly created workspaces, as this interferesthe least with existing go tools.

In full isolation mode, package imports will only search in the packages thatare installed inside the workspace.This has some advantages:

1. Tools such as IDE's don't have to search the global GOPATH for imports, whichcan result in a significant speedup for operations such as indexing.
2. You always know the location of an imported package.
3. It's not possible to accidentally import of a package that is not managed byyour vendoring tool of choice.

However, there's also some downsides to full isolation of a workspace. These areall caused by the fact that the project you're actually working on is not insideyourGOPATH anymore. So normally go would not be able to find any importsto it. This is partially worked around by locally installing the project intoyour workspace, but it does not fix all issues.

In the sections below the remaining issues are described and you can decide foryourself if the above advantages are worth the disadvantages. If you want to tryout full isolation you can create a new workspace using the--full-isolationflag:

$ vg init --full-isolation# To change an existing workspace, you have to destroy and recreate it$ vg destroy example$ vg activate example --full-isolation

This will cause the workspace to use full isolation import mode each time it isactivated in the future. So there's no need to specify the--full-isolation flag on each activation afterwards.

If you havebindfs installed the issues you will runinto are only a slight inconvenience, for which easy workarounds exist. However,it is important that you know about them, because they will probably causeconfusion otherwise. If you run into any other issues than the ones mentionedhere,please report them.

The first set of issues happen when using relative reference to packages incommands. Some examples of this are:

• go list ./... will return weirdly formatted paths, suchas_/home/stream/go/src/github.com/GetStream/vg.
• go test ./..., might cause aninit function to be executed twice.
• go build ./... won't work when aninternal package is present in thedirectory. Here you can expect an error sayinguse of internal package not allowed.

Luckily, this can all easily be worked around by using absolute package pathsfor these commands.So for thevg repo you would use the following alternatives:

# go list ./...go list github.com/GetStream/vg/...# go test ./...gotest github.com/GetStream/vg/...# go build ./...go build github.com/GetStream/vg/...

Another issue that pops up is thatdep doesn't allow it's commands to beexecuted outside of theGOPATH. This is not a problem fordep ensure, sinceyou usually usevg ensure, which handles this automatically. However, this isan issue for other commands, such asdep status anddep init. Luckilythere's an easy workaround for this as well. You can simply usevg globalExec,to execute commands from your regularGOPATH, which fixes the issue:

vg globalExec dep initvg globalExec dep status

Ifbindfs is not installed, symbolic links will be used to do the localinstall.This has the same issues as described forbindfs, but there's also some extraones that cannot be worked around as easily.The reason for this is that go tooling does not like symbolic links inGOPATH(golang/go#15507,golang/go#17451).

Compiling will still work, butgo list github.com/... will not list yourpackage. Other than that there are also issues when usingdelve(#11). Because of these issues itis NOT RECOMMENDED to use virtualgo in full isolation mode withoutbindfsinstalled.

Because virtualgo is just a usability wrapper around changing yourGOPATH fora specific project it is usually quite easy to use it in combination with anIDE. Just check out yourGOPATH after activating a workspace and configure theIDE accordingly. Usually if you show yourGOPATH you will see two pathsseparated by a colon:

$echo$GOPATH/home/stream/.virtualgo/myworkspace:/home/stream/go

If you can set this full string directly that is fine. ForGoLand you have to add the first one first andthen the second one.

When using a workspace in full isolation mode it's even easier to set up asthere's only oneGOPATH set.

$echo$GOPATH/home/stream/.virtualgo/myworkspace

MIT

Would you like to work on cool projects like this? We are currently hiring fortalented Gophers in Amsterdam and Boulder, get in touch with us if you areinterested!tommaso@getstream.io