- Notifications
You must be signed in to change notification settings - Fork2.1k
Go configuration with fangs
License
spf13/viper
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Viper is heading towards v2 and we would love to hear whatyou wouldlike to see in it. Share your thoughts here:https://forms.gle/R6faU74qPRPAzchZ9
Thank you!
Go configuration with fangs!
Many Go projects are built using Viper including:
- Hugo
- EMC RexRay
- Imgur’s Incus
- Nanobox/Nanopack
- Docker Notary
- BloomApi
- doctl
- Clairctl
- Mercure
- Meshery
- Bearer
- Coder
- Vitess
go get github.com/spf13/viper
NOTE Viper usesGo Modules to manage dependencies.
Viper is a complete configuration solution for Go applications including12-Factor apps. It is designed towork within any application, and can handle all types of configuration needsand formats. It supports:
- setting defaults
- setting explicit values
- reading config files
- dynamic discovery of config files across multiple locations
- reading from environment variables
- reading from remote systems (e.g. Etcd or Consul)
- reading from command line flags
- reading from buffers
- live watching and updating configuration
- aliasing configuration keys for easy refactoring
Viper can be thought of as a registry for all of your applications'configuration needs.
Viper can read from multiple configuration sources and merges them togetherinto one set of configuration keys and values.
Viper uses the following precedence for merging:
- explicit call to
Set - flags
- environment variables
- config files
- external key/value stores
- defaults
NOTE Viper configuration keys are case insensitive.
Viper requires minimal configuration to load config files. Viper currently supports:
- JSON
- TOML
- YAML
- INI
- envfile
- Java Propeties
A single Viper instance only supports a single configuration file, but multiplepaths may be searched for one.
Here is an example of how to use Viper to search for and read a configurationfile. At least one path should be provided where a configuration file isexpected.
// Name of the config file without an extension (Viper will intuit the type// from an extension on the actual file)viper.SetConfigName("config")// Add search paths to find the fileviper.AddConfigPath("/etc/appname/")viper.AddConfigPath("$HOME/.appname")viper.AddConfigPath(".")// Find and read the config fileerr:=viper.ReadInConfig()// Handle errorsiferr!=nil {panic(fmt.Errorf("fatal error config file: %w",err))}
You can handle the specific case where no config file is found.
varfileLookupError viper.FileLookupErroriferr:=viper.ReadInConfig();err!=nil {iferrors.As(err,&fileLookupError) {// Indicates an explicitly set config file is not found (such as with// using `viper.SetConfigFile`) or that no config file was found in// any search path (such as when using `viper.AddConfigPath`) }else {// Config file was found but another error was produced }}// Config file found and successfully parsed
NOTE (since 1.6) You can also have a file without an extension andspecify the format programmatically, which is useful for files that naturallyhave no extension (e.g.,
.bashrc).
At times you may want to store all configuration modifications made during runtime.
// Writes current config to the path set by `AddConfigPath` and `SetConfigName`viper.WriteConfig()viper.SafeWriteConfig()// Like the above, but will error if the config file exists// Writes current config to a specific placeviper.WriteConfigAs("/path/to/my/.config")// Will error since it has already been writtenviper.SafeWriteConfigAs("/path/to/my/.config")viper.SafeWriteConfigAs("/path/to/my/.other_config")
As a rule of the thumb, methods prefixed withSafe won't overwrite anyexisting file, while other methods will.
Gone are the days of needing to restart a server to have a config takeeffect--Viper powered applications can read an update to a config file whilerunning and not miss a beat.
It's also possible to provide a function for Viper to run each time a changeoccurs.
// All config paths must be defined prior to calling `WatchConfig()`viper.AddConfigPath("$HOME/.appname")viper.OnConfigChange(func(e fsnotify.Event) {fmt.Println("Config file changed:",e.Name)})viper.WatchConfig()
Viper predefines many configuration sources but you can also implement your ownrequired configuration source.
viper.SetConfigType("yaml")varyamlExample= []byte(`hacker: truehobbies:- skateboarding- snowboarding- goname: steve`)viper.ReadConfig(bytes.NewBuffer(yamlExample))viper.Get("name")// "steve"
A good configuration system will support default values, which are used if akey hasn't been set in some other way.
Examples:
viper.SetDefault("ContentDir","content")viper.SetDefault("LayoutDir","layouts")viper.SetDefault("Taxonomies",map[string]string{"tag":"tags","category":"categories"})
Viper allows explict setting of configuration, such as from your ownapplication logic.
viper.Set("verbose",true)viper.Set("host.port",5899)// Set an embedded key
Aliases permit a single value to be referenced by multiple keys
viper.RegisterAlias("loud","Verbose")viper.Set("verbose",true)// Same result as next lineviper.Set("loud",true)// Same result as prior lineviper.GetBool("loud")// trueviper.GetBool("verbose")// true
Viper has full support for environment variables.
NOTE Unlike other configuration sources, environment variables are casesensitive.
// Tells Viper to use this prefix when reading environment variablesviper.SetEnvPrefix("spf")// Viper will look for "SPF_ID", automatically uppercasing the prefix and keyviper.BindEnv("id")// Alternatively, we can search for any environment variable prefixed and load// them inviper.AutomaticEnv()os.Setenv("SPF_ID","13")id:=viper.Get("id")// 13
- By default, empty environment variables are considered unset and will fall back tothe next configuration source, unless
AllowEmptyEnvis used. - Viper does not "cache" environment variables--the value will be read eachtime it is accessed.
SetEnvKeyReplacerandEnvKeyReplacerallow you to rewrite environmentvariable keys, which is useful to merge SCREAMING_SNAKE_CASE environmentvariables with kebab-cased configuration values from other sources.
Viper has the ability to bind to flags. Specifically, Viper supportspflag as used in theCobra library.
Like environment variables, the value is not set when the binding method iscalled, but when it is accessed.
For individual flags, theBindPFlag method provides this functionality.
serverCmd.Flags().Int("port",1138,"Port to run Application server on")viper.BindPFlag("port",serverCmd.Flags().Lookup("port"))
You can also bind an existing set of pflags.
pflag.Int("flagname",1234,"help message for flagname")pflag.Parse()viper.BindPFlags(pflag.CommandLine)i:=viper.GetInt("flagname")// Retrieve values from viper instead of pflag
The standard libraryflag package is notdirectly supported, but may be parsed through pflag.
package mainimport ("flag""github.com/spf13/pflag")funcmain() {// Using standard library "flag" packageflag.Int("flagname",1234,"help message for flagname")// Pass standard library flags to pflagpflag.CommandLine.AddGoFlagSet(flag.CommandLine)pflag.Parse()// Viper takes overviper.BindPFlags(pflag.CommandLine)}
Use of pflag may be avoided entirely by implementing theFlagValue andFlagValueSet interfaces.
// Implementing FlagValuetypemyFlagstruct {}func (fmyFlag)HasChanged()bool {returnfalse }func (fmyFlag)Name()string {return"my-flag-name" }func (fmyFlag)ValueString()string {return"my-flag-value" }func (fmyFlag)ValueType()string {return"string" }viper.BindFlagValue("my-flag-name",myFlag{})// Implementing FlagValueSettypemyFlagSetstruct {flags []myFlag}func (fmyFlagSet)VisitAll(fnfunc(FlagValue)) {for_,flag:=rangeflags {fn(flag)}}fSet:=myFlagSet{flags: []myFlag{myFlag{},myFlag{}},}viper.BindFlagValues("my-flags",fSet)
To enable remote support in Viper, do a blank import of theviper/remotepackage.
import _"github.com/spf13/viper/remote"
Viper supports the following remote key/value stores. Examples for each areprovided below.
- Etcd and Etcd3
- Consul
- Firestore
- NATS
Viper will read a config string retrieved from a path in a key/value store.
Viper supports multiple hosts separated by;. For example:http://127.0.0.1:4001;http://127.0.0.1:4002.
Viper usescrypt to retrieveconfiguration from the key/value store, which means that you can store yourconfiguration values encrypted and have them automatically decrypted if youhave the correct GPG keyring. Encryption is optional.
Crypt has a command-line helper that you can use to put configurations in yourkey/value store.
$ go get github.com/sagikazarmark/crypt/bin/crypt$ cryptset -plaintext /config/hugo.json /Users/hugo/settings/config.json$ crypt get -plaintext /config/hugo.jsonSee the Crypt documentation for examples of how to set encrypted values, orhow to use Consul.
viper.AddRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json")viper.SetConfigType("json")// because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv"err:=viper.ReadRemoteConfig()
viper.AddRemoteProvider("etcd3","http://127.0.0.1:4001","/config/hugo.json")viper.SetConfigType("json")// because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv"err:=viper.ReadRemoteConfig()
Given a Consul keyMY_CONSUL_KEY with the value:
{"port":8080,"hostname":"myhostname.com"}viper.AddRemoteProvider("consul","localhost:8500","MY_CONSUL_KEY")viper.SetConfigType("json")// Need to explicitly set this to jsonerr:=viper.ReadRemoteConfig()fmt.Println(viper.Get("port"))// 8080
viper.AddRemoteProvider("firestore","google-cloud-project-id","collection/document")viper.SetConfigType("json")// Config's format: "json", "toml", "yaml", "yml"err:=viper.ReadRemoteConfig()
Of course, you're allowed to useSecureRemoteProvider also.
viper.AddRemoteProvider("nats","nats://127.0.0.1:4222","myapp.config")viper.SetConfigType("json")err:=viper.ReadRemoteConfig()
viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg")viper.SetConfigType("json")// because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv"err:=viper.ReadRemoteConfig()
// Alternatively, you can create a new viper instancevarruntime_viper=viper.New()runtime_viper.AddRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.yml")runtime_viper.SetConfigType("yaml")// because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv"// Read from remote config the first timeerr:=runtime_viper.ReadRemoteConfig()// Unmarshal configruntime_viper.Unmarshal(&runtime_conf)// Open a goroutine to watch remote changes forevergofunc(){for {time.Sleep(time.Second*5)// delay after each request// Currently, only tested with Etcd supporterr:=runtime_viper.WatchRemoteConfig()iferr!=nil {log.Errorf("unable to read remote config: %v",err)continue}// Unmarshal new config into our runtime config structruntime_viper.Unmarshal(&runtime_conf)}}()
The simplest way to retrieve configuration values from Viper is to useGet*functions.Get will return an any type, but specific types may be retrievedwithGet<Type> functions.
Note that eachGet* function will return a zero value if it’s key is notfound. To check if a key exists, use theIsSet method.
Nested keys use. as a delimiter and numbers for array indexes. Given thefollowing configuration:
{"datastore": {"metric": {"host":"127.0.0.1","ports": [5799,6029 ] } }}GetString("datastore.metric.host")// "127.0.0.1"GetInt("host.ports.1")// 6029
NOTE Viperdoes not deep merge configuration values. Complex valuesthat are overridden will be entirely replaced.
If there exists a key that matches the delimited key path, its value will bereturned instead.
{"datastore.metric.host":"0.0.0.0","datastore": {"metric": {"host":"127.0.0.1" } }}GetString("datastore.metric.host")// "0.0.0.0"
It's often useful to extract a subset of configuration (e.g., when developing areusable module which should accept specific sections of configuration).
cache:cache1:item-size:64max-items:100cache2:item-size:80max-items:200
funcNewCache(v*Viper)*Cache {return&Cache{ItemSize:v.GetInt("item-size"),MaxItems:v.GetInt("max-items"),}}cache1Config:=viper.Sub("cache.cache1")ifcache1Config==nil {// Sub returns nil if the key cannot be foundpanic("cache configuration not found")}cache1:=NewCache(cache1Config)
You also have the option of unmarshaling configuration to a struct, map, etc.,usingUnmarshal* methods.
typeconfigstruct {PortintNamestringPathMapstring`mapstructure:"path_map"`}varCconfigerr:=viper.Unmarshal(&C)iferr!=nil {t.Fatalf("unable to decode into struct, %v",err)}
If you want to unmarshal configuration where the keys themselves contain.(the default key delimiter), you can change the delimiter.
v:=viper.NewWithOptions(viper.KeyDelimiter("::"))v.SetDefault("chart::values",map[string]any{"ingress":map[string]any{"annotations":map[string]any{"traefik.frontend.rule.type":"PathPrefix","traefik.ingress.kubernetes.io/ssl-redirect":"true",},},})typeconfigstruct {Chartstruct{Valuesmap[string]any}}varCconfigv.Unmarshal(&C)
Viper also supports unmarshaling into embedded structs.
/*Example config:module: enabled: true token: 89h3f98hbwf987h3f98wenf89ehf*/typeconfigstruct {Modulestruct {EnabledboolmoduleConfig`mapstructure:",squash"`}}typemoduleConfigstruct {Tokenstring}varCconfigerr:=viper.Unmarshal(&C)iferr!=nil {t.Fatalf("unable to decode into struct, %v",err)}
Viper usesgithub.com/go-viper/mapstructureunder the hood for unmarshaling values which usesmapstructure tags, bydefault.
You may need to marshal all the settings held in Viper into a string. You canuse your favorite format's marshaller with the config returned byAllSettings.
import (yaml"go.yaml.in/yaml/v3")funcyamlStringSettings()string {c:=viper.AllSettings()bs,err:=yaml.Marshal(c)iferr!=nil {log.Fatalf("unable to marshal config to YAML: %v",err)}returnstring(bs)}
A frequently requested feature is adding more value formats and decoders (forexample; parsing character delimited strings into slices. This is alreadyavailable in Viper using mapstructure decode hooks.
Read more inthis blogpost.
Viper is designed to be acompanion toCobra. While both can operate completelyindependently, together they make a powerful pair to handle much of yourapplication foundation needs.
Yes, but there are two things to be aware of.
- The Viper project is currently prioritizing backwards compatibility andstability over features.
- Features may be deferred until Viper 2 forms.
tl;dr: Yes.
Each will have its own unique configuration and can read from a differentconfiguration source. All of the functions that the Viper package supports aremirrored as methods on a Viper instance.
x:=viper.New()y:=viper.New()x.SetDefault("ContentDir","content")y.SetDefault("ContentDir","foobar")
The best practice is to initialize a Viper instance and pass that around whennecessary.
Viper comes with a global instance (singleton) out of the box. Although itmakes setting up configuration easy, using it is generally discouraged as itmakes testing harder and can lead to unexpected behavior.
The global instance may be deprecated in the future. See#1855 for more details.
tl;dr: No.
Viper merges configuration from various sources, many of which are either caseinsensitive or use different casing than other sources (e.g., env vars). Inorder to provide the best experience when using multiple sources, all keys aremade case insensitive.
There has been several attempts to implement case sensitivity, butunfortunately it's not trivial. We might take a stab at implementing it inViper v2, but despite the initialnoise, it does not seem to be requested that much.
You can vote for case sensitivity by filling out this feedback form:https://forms.gle/R6faU74qPRPAzchZ9.
No, you will need to synchronize access to Viper yourself (for example by usingthesync package). Concurrent reads and writes can cause a panic.
For an optimal developer experience, it is recommended to installNix anddirenv.
Alternatively, installGo on your computer then runmake deps to install the rest of the dependencies.
Run the test suite:
maketestRun linters:
make lint# pass -j option to run them in parallelSome linter violations can automatically be fixed:
make fmt
The project is licensed under theMIT License.
About
Go configuration with fangs
Resources
License
Uh oh!
There was an error while loading.Please reload this page.