The Go Blog

Using Go Modules

Tyler Bui-Palsulich and Eno Compton
19 March 2019

Introduction

This post is part 1 in a series.

• Part 1 — Using Go Modules (this post)
• Part 2 —Migrating To Go Modules
• Part 3 —Publishing Go Modules
• Part 4 —Go Modules: v2 and Beyond
• Part 5 —Keeping Your Modules Compatible

Note: For documentation on managing dependencies with modules, seeManaging dependencies.

Go 1.11 and 1.12 include preliminarysupport for modules,Go’snew dependency management systemthat makes dependency version information explicitand easier to manage.This blog post is an introduction to the basic operations neededto get started using modules.

A module is a collection ofGo packagesstored in a file tree with ago.mod file at its root.Thego.mod file defines the module’smodule path,which is also the import path used for the root directory,and itsdependency requirements,which are the other modules needed for a successful build.Each dependency requirement iswritten as a module path and a specificsemantic version.

As of Go 1.11, the go command enables the use of moduleswhen the current directory or any parent directory has ago.mod,provided the directory isoutside$GOPATH/src.(Inside$GOPATH/src, for compatibility, the go commandstill runs in the old GOPATH mode, even if ago.mod is found.See thego command documentationfor details.)Starting in Go 1.13, module mode will be the default for all development.

This post walks through a sequence of common operationsthat arise when developing Go code with modules:

• Creating a new module.
• Adding a dependency.
• Upgrading dependencies.
• Adding a dependency on a new major version.
• Upgrading a dependency to a new major version.
• Removing unused dependencies.

Creating a new module

Let’s create a new module.

Create a new, empty directory somewhere outside$GOPATH/src,cd into that directory, and then create a new source file,hello.go:

package hellofunc Hello() string {    return "Hello, world."}

Let’s write a test, too, inhello_test.go:

package helloimport "testing"func TestHello(t *testing.T) {    want := "Hello, world."    if got := Hello(); got != want {        t.Errorf("Hello() = %q, want %q", got, want)    }}

At this point, the directory contains a package, but not a module,because there is nogo.mod file.If we were working in/home/gopher/hello and rango test now,we’d see:

$ go testPASSok      _/home/gopher/hello 0.020s$

The last line summarizes the overall package test.Because we are working outside$GOPATHand also outside any module,thego command knows no import path forthe current directory and makes up a fake one basedon the directory name:_/home/gopher/hello.

Let’s make the current directory the root of a moduleby usinggo mod init and then trygo test again:

$ go mod init example.com/hellogo: creating new go.mod: module example.com/hello$ go testPASSok      example.com/hello   0.020s$

Congratulations! You’ve written and tested your first module.

Thego mod init command wrote ago.mod file:

$ cat go.modmodule example.com/hellogo 1.12$

Thego.mod file only appears in the root of the module.Packages in subdirectories have import paths consisting ofthe module path plus the path to the subdirectory.For example, if we created a subdirectoryworld,we would not need to (nor want to) rungo mod init there.The package would automatically be recognized as part of theexample.com/hello module, with import pathexample.com/hello/world.

Adding a dependency

The primary motivation for Go modules was to improve theexperience of using (that is, adding a dependency on)code written by other developers.

Let’s update ourhello.go to importrsc.io/quoteand use it to implementHello:

package helloimport "rsc.io/quote"func Hello() string {    return quote.Hello()}

Now let’s run the test again:

$ go testgo: finding rsc.io/quote v1.5.2go: downloading rsc.io/quote v1.5.2go: extracting rsc.io/quote v1.5.2go: finding rsc.io/sampler v1.3.0go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0cgo: downloading rsc.io/sampler v1.3.0go: extracting rsc.io/sampler v1.3.0go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0cgo: extracting golang.org/x/text v0.0.0-20170915032832-14c0d48ead0cPASSok      example.com/hello   0.023s$

Thego command resolves imports by using the specificdependency module versions listed ingo.mod.When it encounters animport of a package not providedby any module ingo.mod, thego command automaticallylooks up the module containing that package and adds it togo.mod, using the latest version.(“Latest” is defined as thelatest tagged stable (non-prerelease) version,or else the latest tagged prerelease version,or else the latest untagged version.)In our example,go test resolved the new importrsc.io/quoteto the modulersc.io/quote v1.5.2.It also downloaded two dependencies used byrsc.io/quote,namelyrsc.io/sampler andgolang.org/x/text.Only direct dependencies are recorded in thego.mod file:

$ cat go.modmodule example.com/hellogo 1.12require rsc.io/quote v1.5.2$

A secondgo test command will not repeat this work,since thego.mod is now up-to-date and the downloadedmodules are cached locally (in$GOPATH/pkg/mod):

$ go testPASSok      example.com/hello   0.020s$

Note that while thego command makes adding a new dependencyquick and easy, it is not without cost.Your module now literallydepends on the new dependencyin critical areas such as correctness, security, and proper licensing,just to name a few.For more considerations, see Russ Cox’s blog post,“Our Software Dependency Problem.”

As we saw above, adding one direct dependency oftenbrings in other indirect dependencies too.The commandgo list -m all lists the current moduleand all its dependencies:

$ go list -m allexample.com/hellogolang.org/x/text v0.0.0-20170915032832-14c0d48ead0crsc.io/quote v1.5.2rsc.io/sampler v1.3.0$

In thego list output, the current module,also known as themain module,is always the first line,followed by dependencies sorted by module path.

Thegolang.org/x/text versionv0.0.0-20170915032832-14c0d48ead0cis an example of apseudo-version,which is thego command’s version syntaxfor a specific untagged commit.

In addition togo.mod, thego commandmaintains a file namedgo.sum containingthe expectedcryptographic hashes of the content of specific module versions:

$ cat go.sumgolang.org/x/text v0.0.0-20170915032832-14c0d48ead0c h1:qgOY6WgZO...golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:Nq...rsc.io/quote v1.5.2 h1:w5fcysjrx7yqtD/aO+QwRjYZOKnaM9Uh2b40tElTs3...rsc.io/quote v1.5.2/go.mod h1:LzX7hefJvL54yjefDEDHNONDjII0t9xZLPX...rsc.io/sampler v1.3.0 h1:7uVkIFmeBqHfdjD+gZwtXXI+RODJ2Wc4O7MPEh/Q...rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9...$

Thego command uses thego.sum file to ensure thatfuture downloads of these modules retrieve the same bitsas the first download,to ensure the modules your project depends ondo not change unexpectedly,whether for malicious, accidental, or other reasons.Bothgo.mod andgo.sum should be checked into version control.

Upgrading dependencies

With Go modules, versions are referenced with semantic version tags.A semantic version has three parts: major, minor, and patch.For example, forv0.1.2, the major version is 0, the minor version is 1,and the patch version is 2.Let’s walk through a couple minor version upgrades.In the next section, we’ll consider a major version upgrade.

From the output ofgo list -m all,we can see we’re using an untagged version ofgolang.org/x/text.Let’s upgrade to the latest tagged version and test that everything still works:

$ go get golang.org/x/textgo: finding golang.org/x/text v0.3.0go: downloading golang.org/x/text v0.3.0go: extracting golang.org/x/text v0.3.0$ go testPASSok      example.com/hello   0.013s$

Woohoo! Everything passes.Let’s take another look atgo list -m all and thego.mod file:

$ go list -m allexample.com/hellogolang.org/x/text v0.3.0rsc.io/quote v1.5.2rsc.io/sampler v1.3.0$ cat go.modmodule example.com/hellogo 1.12require (    golang.org/x/text v0.3.0 // indirect    rsc.io/quote v1.5.2)$

Thegolang.org/x/text package has been upgraded to the latest tagged version (v0.3.0).Thego.mod file has been updated to specifyv0.3.0 too.Theindirect comment indicates a dependency is not used directlyby this module, only indirectly by other module dependencies.Seego help modules for details.

Now let’s try upgrading thersc.io/sampler minor version.Start the same way, by runninggo get and running tests:

$ go get rsc.io/samplergo: finding rsc.io/sampler v1.99.99go: downloading rsc.io/sampler v1.99.99go: extracting rsc.io/sampler v1.99.99$ go test--- FAIL: TestHello (0.00s)    hello_test.go:8: Hello() = "99 bottles of beer on the wall, 99 bottles of beer, ...", want "Hello, world."FAILexit status 1FAIL    example.com/hello   0.014s$

Uh, oh! The test failure shows that thelatest version ofrsc.io/sampler is incompatible with our usage.Let’s list the available tagged versions of that module:

$ go list -m -versions rsc.io/samplerrsc.io/sampler v1.0.0 v1.2.0 v1.2.1 v1.3.0 v1.3.1 v1.99.99$

We had been using v1.3.0; v1.99.99 is clearly no good.Maybe we can try using v1.3.1 instead:

$ go get rsc.io/sampler@v1.3.1go: finding rsc.io/sampler v1.3.1go: downloading rsc.io/sampler v1.3.1go: extracting rsc.io/sampler v1.3.1$ go testPASSok      example.com/hello   0.022s$

Note the explicit@v1.3.1 in thego get argument.In general each argument passed togo get can takean explicit version; the default is@latest,which resolves to the latest version as defined earlier.

Adding a dependency on a new major version

Let’s add a new function to our package:func Proverb returns a Go concurrency proverb,by callingquote.Concurrency, which is provided bythe modulersc.io/quote/v3.First we updatehello.go to add the new function:

package helloimport (    "rsc.io/quote"    quoteV3 "rsc.io/quote/v3")func Hello() string {    return quote.Hello()}func Proverb() string {    return quoteV3.Concurrency()}

Then we add a test tohello_test.go:

func TestProverb(t *testing.T) {    want := "Concurrency is not parallelism."    if got := Proverb(); got != want {        t.Errorf("Proverb() = %q, want %q", got, want)    }}

Then we can test our code:

$ go testgo: finding rsc.io/quote/v3 v3.1.0go: downloading rsc.io/quote/v3 v3.1.0go: extracting rsc.io/quote/v3 v3.1.0PASSok      example.com/hello   0.024s$

Note that our module now depends on bothrsc.io/quote andrsc.io/quote/v3:

$ go list -m rsc.io/q...rsc.io/quote v1.5.2rsc.io/quote/v3 v3.1.0$

Each different major version (v1,v2, and so on) of a Go moduleuses a different module path: starting atv2, the path must end in the major version.In the example,v3 ofrsc.io/quote is no longerrsc.io/quote: instead,it is identified by the module pathrsc.io/quote/v3.This convention is calledsemantic import versioning,and it gives incompatible packages (those with different major versions)different names.In contrast,v1.6.0 ofrsc.io/quote should be backwards-compatiblewithv1.5.2, so it reuses the namersc.io/quote.(In the previous section,rsc.io/samplerv1.99.99should have been backwards-compatiblewithrsc.io/samplerv1.3.0, but bugs or incorrect client assumptions aboutmodule behavior can both happen.)

Thego command allows a build to include at most one version ofany particular module path, meaning at most one of each majorversion: onersc.io/quote, onersc.io/quote/v2, onersc.io/quote/v3,and so on.This gives module authors a clear rule about possible duplicationof a single module path: it is impossible for a program to build with bothrsc.io/quote v1.5.2 andrsc.io/quote v1.6.0.At the same time, allowing different major versions of a module(because they have different paths)gives module consumers the ability toupgrade to a new major version incrementally.In this example, we wanted to usequote.Concurrency fromrsc/quote/v3 v3.1.0but are not yet ready to migrate our uses ofrsc.io/quote v1.5.2.The ability to migrate incrementallyis especially important in a large program or codebase.

Upgrading a dependency to a new major version

Let’s complete our conversion from usingrsc.io/quote to using onlyrsc.io/quote/v3.Because of the major version change, we should expect that some APIs may havebeen removed, renamed, or otherwise changed in incompatible ways.Reading the docs, we can see thatHello has becomeHelloV3:

$ go doc rsc.io/quote/v3package quote // import "rsc.io/quote/v3"Package quote collects pithy sayings.func Concurrency() stringfunc GlassV3() stringfunc GoV3() stringfunc HelloV3() stringfunc OptV3() string$

We can update our use ofquote.Hello() inhello.go to usequoteV3.HelloV3():

package helloimport quoteV3 "rsc.io/quote/v3"func Hello() string {    return quoteV3.HelloV3()}func Proverb() string {    return quoteV3.Concurrency()}

And then at this point, there’s no need for the renamed import anymore,so we can undo that:

package helloimport "rsc.io/quote/v3"func Hello() string {    return quote.HelloV3()}func Proverb() string {    return quote.Concurrency()}

Let’s re-run the tests to make sure everything is working:

$ go testPASSok      example.com/hello       0.014s

Removing unused dependencies

We’ve removed all our uses ofrsc.io/quote,but it still shows up ingo list -m all and in ourgo.mod file:

$ go list -m allexample.com/hellogolang.org/x/text v0.3.0rsc.io/quote v1.5.2rsc.io/quote/v3 v3.1.0rsc.io/sampler v1.3.1$ cat go.modmodule example.com/hellogo 1.12require (    golang.org/x/text v0.3.0 // indirect    rsc.io/quote v1.5.2    rsc.io/quote/v3 v3.0.0    rsc.io/sampler v1.3.1 // indirect)$

Why? Because building a single package, like withgo build orgo test,can easily tell when something is missing and needs to be added,but not when something can safely be removed.Removing a dependency can only be done afterchecking all packages in a module,and all possible build tag combinations for those packages.An ordinary build command does not load this information,and so it cannot safely remove dependencies.

Thego mod tidy command cleans up these unused dependencies:

$ go mod tidy$ go list -m allexample.com/hellogolang.org/x/text v0.3.0rsc.io/quote/v3 v3.1.0rsc.io/sampler v1.3.1$ cat go.modmodule example.com/hellogo 1.12require (    golang.org/x/text v0.3.0 // indirect    rsc.io/quote/v3 v3.1.0    rsc.io/sampler v1.3.1 // indirect)$ go testPASSok      example.com/hello   0.020s$

Conclusion

Go modules are the future of dependency management in Go.Module functionality is now available in all supported Go versions(that is, in Go 1.11 and Go 1.12).

This post introduced these workflows using Go modules:

• go mod init creates a new module, initializing thego.mod file that describes it.
• go build,go test, and other package-building commands add new dependencies togo.mod as needed.
• go list -m all prints the current module’s dependencies.
• go get changes the required version of a dependency (or adds a new dependency).
• go mod tidy removes unused dependencies.

We encourage you to start using modules in your local developmentand to addgo.mod andgo.sum files to your projects.To provide feedback and help shape the future of dependency management in Go,please send usbug reports orexperience reports.

Thanks for all your feedback and help improving modules.

Next article:Debugging what you deploy in Go 1.12
Previous article:The New Go Developer Network
Blog Index