- Notifications
You must be signed in to change notification settings - Fork978
configuration library for JVM languages using HOCON files
lightbend/config
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
- implemented in plain Java with no dependencies
- supports files in three formats: Java properties, JSON, and ahuman-friendly JSON superset
- merges multiple files across all formats
- can load from files, URLs, or classpath
- good support for "nesting" (treat any subtree of the config thesame as the whole config)
- users can override the config with Java system properties,
java -Dmyapp.foo.bar=10 - supports configuring an app, with its framework and libraries,all from a single file such as
application.conf - parses duration and size settings, "512k" or "10 seconds"
- converts types, so if you ask for a boolean and the valueis the string "yes", or you ask for a float and the value isan int, it will figure it out.
- JSON superset features:
- comments
- includes
- substitutions (
"foo" : ${bar},"foo" : Hello ${who}) - properties-like notation (
a.b=c) - less noisy, more lenient syntax
- substitute environment variables (
logdir=${HOME}/logs)
- API based on immutable
Configinstances, for thread safetyand easy reasoning about config transformations - extensive test coverage
This library limits itself to config files. If you want to loadconfig from a database or something, you would need to write somecustom code. The library has nice support for mergingconfigurations so if you build one from a custom source it's easyto merge it in.
Table of Contentsgenerated withDocToc
- Essential Information
- Using the Library
- Using HOCON, the JSON Superset
- Miscellaneous Notes
Typesafe Config is compatible with Java 8 and above.
You can find published releases on Maven Central.
<dependency> <groupId>com.typesafe</groupId> <artifactId>config</artifactId> <version>1.4.4</version></dependency>sbt dependency:
libraryDependencies += "com.typesafe" % "config" % "1.4.4"Link for direct download if you don't use a dependency manager:
Please see NEWS.md in this directory,https://github.com/lightbend/config/blob/main/NEWS.md
- Online:https://lightbend.github.io/config/latest/api/
- also published in jar form
- consider reading this README first for an intro
- for questions about the
.conffile format, readHOCON.mdin this directory
NOTE: Please readReadme #Maintained-by before spending time suggesting changes to this library.
Report bugs to the GitHub issue tracker. Send patches as pullrequests on GitHub.
Before we can accept pull requests, you will need to agree to theTypesafe Contributor License Agreement online, using your GitHubaccount - it takes 30 seconds. You can do this athttps://www.lightbend.com/contribute/cla
Please seeCONTRIBUTINGfor more including how to make a release.
The build uses sbt and the tests are written in Scala; however,the library itself is plain Java and the published jar has noScala dependency.
import com.typesafe.config.ConfigFactoryConfig conf = ConfigFactory.load();int bar1 = conf.getInt("foo.bar");Config foo = conf.getConfig("foo");int bar2 = foo.getInt("bar");See the examples in theexamples/directory.
You can run these from the sbt console with the commandsproject config-simple-app-java and thenrun.
In brief, as shown in the examples:
- libraries should use a
Configinstance provided by the app,if any, and useConfigFactory.load()if no specialConfigis provided. Libraries should put their defaults in areference.confon the classpath. - apps can create a
Confighowever they want(ConfigFactory.load()is easiest and least-surprising), thenprovide it to their libraries. AConfigcan be created withthe parser methods inConfigFactoryor built up from any fileformat or data source you like with the methods inConfigValueFactory.
Objects are immutable, so methods onConfig which transform theconfiguration return a newConfig. Other types such asConfigParseOptions,ConfigResolveOptions,ConfigObject,etc. are also immutable. See theAPI docs fordetails of course.
There isn't a schema language or anything like that. However, twosuggested tools are:
- use thecheckValid() method
- access your config through a Settings class with a field foreach setting, and instantiate it on startup (immediatelythrowing an exception if any settings are missing)
In Scala, a Settings class might look like:
class Settings(config: Config) { // validate vs. reference.conf config.checkValid(ConfigFactory.defaultReference(), "simple-lib") // non-lazy fields, we want all exceptions at construct time val foo = config.getString("simple-lib.foo") val bar = config.getInt("simple-lib.bar")}See the examples/ directory for a full compilable program usingthis pattern.
The convenience methodConfigFactory.load() loads the following(first-listed are higher priority):
- system properties
application.conf(all resources on classpath with this name)application.json(all resources on classpath with this name)application.properties(all resources on classpath with thisname)reference.conf(all resources on classpath with this name)
The idea is that libraries and frameworks should ship with areference.conf in their jar. Applications should provide anapplication.conf, or if they want to create multipleconfigurations in a single JVM, they could useConfigFactory.load("myapp") to load their ownmyapp.conf.
Libraries and frameworks should default toConfigFactory.load()if the application does not provide a customConfig object. Thisway, libraries will see configuration fromapplication.conf andusers can configure the whole app, with its libraries, in a singleapplication.conf file.
Libraries and frameworks should also allow the application toprovide a customConfig object to be used instead of thedefault, in case the application needs multiple configurations inone JVM or wants to load extra config files from somewhere. Thelibrary examples inexamples/ show how to accept a custom configwhile defaulting toConfigFactory.load().
For applications usingapplication.{conf,json,properties},system properties can be used to force a different config source(e.g. from command line-Dconfig.file=path/to/config-file):
config.resourcespecifies a resource name - not abasename, i.e.application.confnotapplicationconfig.filespecifies a filesystem path, againit should include the extension, not be a basenameconfig.urlspecifies a URL
Note: you need to pass-Dconfig.file=path/to/config-file before the jar itself, e.g.java -Dconfig.file=path/to/config-file.conf -jar path/to/jar-file.jar. Same applies for-Dconfig.resource=config-file.conf
These system properties specify areplacement forapplication.{conf,json,properties}, not an addition. They onlyaffect apps using the defaultConfigFactory.load()configuration. In the replacement config file, you can useinclude "application" to include the original default configfile; after the include statement you could go on to overridecertain settings.
If you setconfig.resource,config.file, orconfig.urlon-the-fly from inside your program (for example withSystem.setProperty()), be aware thatConfigFactory has someinternal caches and may not see new values for systemproperties. UseConfigFactory.invalidateCaches() to force-reloadsystem properties.
The substitution syntax${foo.bar} will be resolvedtwice. First, all thereference.conf files are merged and thenthe result gets resolved. Second, all theapplication.conf arelayered over the unresolvedreference.conf and the result of thatgets resolved again.
The implication of this is that thereference.conf stack has tobe self-contained; you can't leave an undefined value${foo.bar}to be provided byapplication.conf. It is however possible tooverride a variable thatreference.conf refers to, as long asreference.conf also defines that variable itself.
Any two Config objects can be merged with an associative operationcalledwithFallback, likemerged = firstConfig.withFallback(secondConfig).
ThewithFallback operation is used inside the library to mergeduplicate keys in the same file and to merge multiple files.ConfigFactory.load() uses it to stack system properties overapplication.conf overreference.conf.
You can also usewithFallback to merge in some hardcoded values,or to "lift" a subtree up to the root of the configuration; sayyou have something like:
foo=42dev.foo=57prod.foo=10Then you could code something like:
Config devConfig = originalConfig .getConfig("dev") .withFallback(originalConfig)There are lots of ways to usewithFallback.
Many other configuration APIs allow you to provide a default tothe getter methods, like this:
boolean getBoolean(String path, boolean fallback)Here, if the path has no setting, the fallback would bereturned. An API could also returnnull for unset values, so youwould check fornull:
// returns null on unset, check for null and fall backBoolean getBoolean(String path)The methods on theConfig interface do NOT do this, for twomajor reasons:
- If you use a config setting in two places, the defaultfallback value gets cut-and-pasted and typically out ofsync. This can result in Very Evil Bugs.
- If the getter returns
null(orNone, in Scala) then everytime you get a setting you have to write handling code fornull/Noneand that code will almost always just throw anexception. Perhaps more commonly, people forget to check fornullat all, so missing settings result inNullPointerException.
For most situations, failure to have a setting is simply a bug to fix(in either code or the deployment environment). Therefore, if asetting is unset, by default the getters on theConfig interfacethrow an exception.
If you want to allow a setting to be missing fromapplication.conf in a particular case, then here are someoptions:
- Set it in a
reference.confincluded in your library orapplication jar, so there's a default value. - Use the
Config.hasPath()method to check in advance whetherthe path exists (rather than checking fornull/Noneafter asyou might in other APIs). - Catch and handle
ConfigException.Missing. NOTE: using anexception for control flow like this is much slower than usingConfig.hasPath(); the JVM has to do a lot of work to throwan exception. - In your initialization code, generate a
Configwith yourdefaults in it (using something likeConfigFactory.parseMap())then fold that default config into your loaded config usingwithFallback(), and use the combined config in yourprogram. "Inlining" your reference config in the code like thisis probably less convenient than using areference.conffile,but there may be reasons to do it. - Use
Config.root()to get theConfigObjectfor theConfig;ConfigObjectimplementsjava.util.Map<String,?>andtheget()method onMapreturns null for missing keys. Seethe API docs for more detail onConfigvs.ConfigObject. - Set the setting to
nullinreference.conf, then useConfig.getIsNullandConfig.hasPathOrNullto handlenullin a special way while still throwing an exception if the settingis entirely absent.
Therecommended path (for most cases, in most apps) is that yourequire all settings to be present in eitherreference.conf orapplication.conf and allowConfigException.Missing to bethrown if they are not. That's the design intent of theConfigAPI design.
Consider the "Settings class" pattern withcheckValid() toverify that you have all settings when you initialize theapp. See theSchemas and Validationsection of this README for more details on this pattern.
If you do need a setting to be optional: checkinghasPath() inadvance should be the same amount of code (in Java) as checkingfornull afterward, without the risk ofNullPointerExceptionwhen you forget. In Scala, you could write an enrichment classlike this to use the idiomaticOption syntax:
implicitclassRichConfig(valunderlying:Config)extendsAnyVal {defgetOptionalBoolean(path:String):Option[Boolean]=if (underlying.hasPath(path)) {Some(underlying.getBoolean(path)) }else {None }}
Since this library is a Java library it doesn't come with that outof the box, of course.
It is understood that sometimes defaults in code make sense. Forexample, if your configuration lets users invent new sections, youmay not have all paths up front and may be unable to set updefaults inreference.conf for dynamic paths. The design intentofConfig isn't toprohibit inline defaults, but simply torecognize that it seems to be the 10% case (rather than the 90%case). Even in cases where dynamic defaults are needed, you mayfind that usingwithFallback() to build a completenothing-missingConfig in one central place in your code keepsthings tidy.
Whatever you do, please remember not to cut-and-paste defaultvalues into multiple places in your code. You have been warned!:-)
To read and modify configuration, you'll use theConfiginterface. AConfig looks at a JSON-equivalent data structure asa one-level map from paths to values. So if your JSON looks likethis:
"foo" : { "bar" : 42 "baz" : 43 }Using theConfig interface, you could writeconf.getInt("foo.bar"). Thefoo.bar string is called apathexpression(HOCON.mdhas the syntax details for these expressions). Iterating over thisConfig, you would get two entries;"foo.bar" : 42 and"foo.baz" : 43. When iterating aConfig you will not findnestedConfig (because everything gets flattened into onelevel).
When looking at a JSON tree as aConfig,null values aretreated as if they were missing. Iterating over aConfig willskipnull values.
You can also look at aConfig in the way most JSON APIs would,through theConfigObjectinterface. This interface represents an object node in the JSONtree.ConfigObject instances come in multi-level trees, and thekeys do not have any syntax (they are just strings, not pathexpressions). Iterating over the above example as aConfigObject, you would get one entry"foo" : { "bar" : 42, "baz" : 43 }, where the value at"foo" is another nestedConfigObject.
InConfigObject,null values are visible (distinct frommissing values), just as they are in JSON.
ConfigObject is a subtype ofConfigValue, where the othersubtypes are the other JSON types (list, string, number, boolean, null).
Config andConfigObject are two ways to look at the sameinternal data structure, and you can convert between them for freeusingConfig.root()andConfigObject.toConfig().
As of version 1.3.0, if you have a Java object that followsJavaBean conventions (zero-args constructor, getters and setters),you can automatically initialize it from aConfig.
UseConfigBeanFactory.create(config.getConfig("subtree-that-matches-bean"), MyBean.class) to do this.
Creating a bean from aConfig automatically validates that theconfig matches the bean's implied schema. Bean fields can beprimitive types, typed lists such asList<Integer>,java.time.Duration,ConfigMemorySize, or even a rawConfig,ConfigObject, orConfigValue (if you'd like to deal with aparticular value manually).
The JSON superset is called "Human-Optimized Config ObjectNotation" or HOCON, and files use the suffix.conf. SeeHOCON.mdin this directory for more detail.
After processing a.conf file, the result is always just a JSONtree that you could have written (less conveniently) in JSON.
- Comments, with
#or// - Allow omitting the
{}around a root object - Allow
=as a synonym for: - Allow omitting the
=or:before a{sofoo { a : 42 } - Allow omitting commas as long as there's a newline
- Allow trailing commas after last element in objects and arrays
- Allow unquoted strings for keys and values
- Unquoted keys can use dot-notation for nested objects,
foo.bar=42meansfoo { bar : 42 } - Duplicate keys are allowed; later values override earlier,except for object-valued keys where the two objects are mergedrecursively
includefeature merges root object in another file intocurrent object, sofoo { include "bar.json" }merges keys inbar.jsoninto the objectfoo- include with no file extension includes any of
.conf,.json,.properties - you can include files, URLs, or classpath resources; use
include url("http://example.com")orfile()orclasspath()syntax to force the type, or use justinclude "whatever"to have the library do what you probably mean(Note:url()/file()/classpath()syntax is not supportedin Play/Akka 2.0, only in later releases.) - substitutions
foo : ${a.b}sets keyfooto the same valueas thebfield in theaobject - substitutions concatenate into unquoted strings,
foo : the quick ${colors.fox} jumped - substitutions fall back to environment variables if they don'tresolve in the config itself, so
${HOME}would work as youexpect. Also, most configs have system properties merged in soyou could use${user.home}. - substitutions normally cause an error if unresolved, butthere is a syntax
${?a.b}to permit them to be missing. +=syntax to append elements to arrays,path += "/bin"- multi-line strings with triple quotes as in Python or Scala
All of these are valid HOCON.
Start with valid JSON:
{ "foo" : { "bar" : 10, "baz" : 12 }}Drop root braces:
"foo" : { "bar" : 10, "baz" : 12}Drop quotes:
foo : { bar : 10, baz : 12}Use= and omit it before{:
foo { bar = 10, baz = 12}Remove commas:
foo { bar = 10 baz = 12}Use dotted notation for unquoted keys:
foo.bar=10foo.baz=12Put the dotted-notation fields on a single line:
foo.bar=10, foo.baz=12The syntax is well-defined (including handling of whitespace andescaping). But it handles many reasonable ways you might want toformat the file.
Note that while you can write HOCON that looks a lot like a Javaproperties file (and many properties files will parse as HOCON),the details of escaping, whitespace handling, comments, and soforth are more like JSON. The spec (see HOCON.md in thisdirectory) has some more detailed notes on this topic.
The${foo.bar} substitution feature lets you avoid cut-and-pastein some nice ways.
This is the obvious use,
standard-timeout = 10msfoo.timeout = ${standard-timeout}bar.timeout = ${standard-timeout}If you duplicate a field with an object value, then the objectsare merged with last-one-wins. So:
foo = { a : 42, c : 5 }foo = { b : 43, c : 6 }means the same as:
foo = { a : 42, b : 43, c : 6 }You can take advantage of this for "inheritance":
data-center-generic = { cluster-size = 6 }data-center-east = ${data-center-generic}data-center-east = { name = "east" }data-center-west = ${data-center-generic}data-center-west = { name = "west", cluster-size = 8 }Usinginclude statements you could split this across multiplefiles, too.
If you put two objects next to each other (close brace of the firston the same line with open brace of the second), they are merged, soa shorter way to write the above "inheritance" example would be:
data-center-generic = { cluster-size = 6 }data-center-east = ${data-center-generic} { name = "east" }data-center-west = ${data-center-generic} { name = "west", cluster-size = 8 }In default uses of the library, exact-match system propertiesalready override the corresponding config properties. However,you can add your own overrides, or allow environment variables tooverride, using the${?foo} substitution syntax.
basedir = "/whatever/whatever"basedir = ${?FORCED_BASEDIR}Here, the override fieldbasedir = ${?FORCED_BASEDIR} simplyvanishes if there's no value forFORCED_BASEDIR, but if you setan environment variableFORCED_BASEDIR for example, it would beused.
A natural extension of this idea is to support several differentenvironment variable names or system property names, if you aren'tsure which one will exist in the target environment.
Object fields and array elements with a${?foo} substitutionvalue just disappear if the substitution is not found:
// this array could have one or two elementspath = [ "a", ${?OPTIONAL_A} ]By setting the JVM property-Dconfig.override_with_env_vars=trueit is possible to override any configuration value using environmentvariables even if an explicit substitution is not specified.
The environment variable value will override any pre-existing valueand also any value provided as Java property.
With this option enabled only environment variables starting withCONFIG_FORCE_ are considered, and the name is mangled as follows:
- the prefix
CONFIG_FORCE_is stripped - single underscore(
_) is converted into a dot(.) - double underscore(
__) is converted into a dash(-) - triple underscore(
___) is converted into a single underscore(_)
i.e. The environment variableCONFIG_FORCE_a_b__c___d set theconfiguration keya.b-c_d
Setting the value of array items from java properties or environmentvariables require specifying the index in the array for the value.So, while in HOCON you can set multiple values into an array orappend to an array:
## HOCONitems = ["a", "b"]items += "c"Using java properties you specify the exact position:
-Ditems.0="a" -Ditems.1="b"as well as with environment variables:
export CONFIG_FORCE_items_0=aexport CONFIG_FORCE_items_1=bValueson the same line are concatenated (for strings andarrays) or merged (for objects).
This is why unquoted strings work, here the number42 and thestringfoo are concatenated into a string42 foo:
key : 42 fooWhen concatenating values into a string, leading and trailingwhitespace is stripped but whitespace between values is kept.
Quoted or unquoted strings can also concatenate with substitutions of course:
tasks-url : ${base-url}/taskstasks-url : ${base-url}"tasks:colon-must-be-quoted"Note: the${} syntax must be outside the quotes!
A concatenation can refer to earlier values of the same field:
path : "/bin"path : ${path}":/usr/bin"Arrays can be concatenated as well:
path : [ "/bin" ]path : ${path} [ "/usr/bin" ]There is a shorthand for appending to arrays:
// equivalent to: path = ${?path} [ "/usr/bin" ]path += "/usr/bin"To prepend or insert into an array, there is no shorthand.
When objects are "concatenated," they are merged, so objectconcatenation is just a shorthand for defining the same objecttwice. The long way (mentioned earlier) is:
data-center-generic = { cluster-size = 6 }data-center-east = ${data-center-generic}data-center-east = { name = "east" }The concatenation-style shortcut is:
data-center-generic = { cluster-size = 6 }data-center-east = ${data-center-generic} { name = "east" }When concatenating objects and arrays, newlines are allowedinside each object or array, but not between them.
Non-newline whitespace is never a field or element separator. So[ 1 2 3 4 ] is an array with one unquoted string element"1 2 3 4". To get an array of four numbers you need either commas ornewlines separating the numbers.
See the spec for full details on concatenation.
Note: Play/Akka 2.0 have an earlier version that supports stringconcatenation, but not object/array concatenation.+= does notwork in Play/Akka 2.0 either. Post-2.0 versions support thesefeatures.
If you have trouble with your configuration, some useful tips.
- Set the Java system property
-Dconfig.trace=loadsto getoutput on stderr describing each file that is loaded.Note: this feature is not included in the older version inPlay/Akka 2.0. - Use
myConfig.root().render()to get aConfigas astring with comments showing where each value came from.This string can be printed out on console or logged toa file etc. - If you see errors like
com.typesafe.config.ConfigException$Missing: No configuration setting found for key foo,and you're sure that key is defined in your config file, they might appear e.g.when you're loading configuration from a thread that's not the JVM's main thread.Try passing theClassLoaderin manually - e.g. withConfigFactory.load(getClass().getClassLoader())or setting the context class loader.If you don't pass one, Lightbend Config uses the calling thread'scontextClassLoader, and in some cases,it may not have your configuration files in its classpath,so loading the config on that thread can yield unexpected, erroneous results.
Currently the library is maintained against Java 8, butversion 1.2.1 and earlier will work with Java 6.
Please use 1.2.1 if you need Java 6 support, though some peoplehave expressed interest in a branch off of 1.3.x supportingJava 7. If you want to work on that branch you might bring it uponchat. We can release ajar for Java 7 if someone(s) steps up to maintain the branch. Themain branch does not use Java 8 "gratuitously" but some APIsthat use Java 8 types will need to be removed.
(For the curious.)
The three file formats each have advantages.
- Java
.properties:- Java standard, built in to JVM
- Supported by many tools such as IDEs
- JSON:
- easy to generate programmatically
- well-defined and standard
- bad for human maintenance, with no way to write comments,and no mechanisms to avoid duplication of similar configsections
- HOCON/
.conf:- nice for humans to read, type, and maintain, with morelenient syntax
- built-in tools to avoid cut-and-paste
- ways to refer to the system environment, such as systemproperties and environment variables
The idea would be to use JSON if you're writing a script to spitout config, and use HOCON if you're maintaining config by hand.If you're doing both, then mix the two.
Two alternatives to HOCON syntax could be:
- YAML is also a JSON superset and has a mechanism for addingcustom types, so the include statements in HOCON could becomea custom type tag like
!include, and substitutions in HOCONcould become a custom tag such as!subst, for example. Theresult is somewhat clunky to write, but would have the samein-memory representation as the HOCON approach. - Put a syntax inside JSON strings, so you might write somethinglike
"$include" : "filename"or allow"foo" : "${bar}".This is a way to tunnel new syntax through a JSON parser, butother than the implementation benefit (using a standard JSONparser), it doesn't really work. It's a bad syntax for humanmaintenance, and it's not valid JSON anymore because properlyinterpreting it requires treating some valid JSON strings assomething other than plain strings. A better approach is toallow mixing true JSON files into the config but also supporta nicer format.
This may not be comprehensive - if you'd like to add mention ofyour wrapper, just send a pull request for this README. We wouldlove to know what you're doing with this library or with the HOCONformat.
- Typesafe Config Guicehttps://github.com/racc/typesafeconfig-guice
- Ficushttps://github.com/iheartradio/ficus
- configzhttps://github.com/arosien/configz
- configshttps://github.com/kxbmap/configs
- config-annotationhttps://github.com/zhongl/config-annotation
- PureConfighttps://github.com/pureconfig/pureconfig
- Simple Scala Confighttps://github.com/ElderResearch/ssc
- konfighttps://github.com/vpon/konfig
- ScalaConfighttps://github.com/andr83/scalaconfig
- static-confighttps://github.com/Krever/static-config
- validated-confighttps://github.com/carlpulley/validated-config
- Cedi Confighttps://github.com/ccadllc/cedi-config
- Cfghttps://github.com/carueda/cfg
- circe-confighttps://github.com/circe/circe-config
- args4chttps://github.com/aaronp/args4c
- beamly-core.confighttps://github.com/beamly/beamly-core.config
- SHoconhttps://github.com/akka-js/shocon (Supports Scala.js and Scala Native)
- sconfighttps://github.com/ekrich/sconfig (Supports JVM, Scala Native, and Scala.js)
- Manage your HOCON configuration files with Puppet!:https://forge.puppetlabs.com/puppetlabs/hocon
- https://github.com/yellowblood/hocon-js (missing features, under development)
- A web based linting toolhttp://www.hoconlint.com/
The license is Apache 2.0, see LICENSE-2.0.txt.
The "Typesafe Config" library is an important foundation to how Akka and other JVM libraries manage configuration. We atLightbend consider the functionality of this library as feature complete. We will make sure "Typesafe Config" keeps up with future JVM versions, but will rarely make any other changes.
We are thankful for all the work@havocp has put into creating the library initially and supporting its users over many more years, even after leaving Lightbend.
About
configuration library for JVM languages using HOCON files
Topics
Resources
Code of conduct
Contributing
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.