Repository files navigation

Package easyjson provides a fast and easy way to marshal/unmarshal Go structsto/from JSON without the use of reflection. In performance tests, easyjsonoutperforms the standardencoding/json package by a factor of 4-5x, and otherJSON encoding packages by a factor of 2-3x.

easyjson aims to keep generated Go code simple enough so that it can be easilyoptimized or fixed. Another goal is to provide users with the ability tocustomize the generated code by providing options not available with thestandardencoding/json package, such as generating "snake_case" names orenablingomitempty behavior by default.

# for Go < 1.17go get -u github.com/mailru/easyjson/...

# for Go >= 1.17go get github.com/mailru/easyjson&& go install github.com/mailru/easyjson/...@latest

easyjson -all<file>.go

The above will generate<file>_easyjson.go containing the appropriate marshaler andunmarshaler funcs for all structs contained in<file>.go.

Please note that easyjson requires a full Go build environment and theGOPATHenvironment variable to be set. This is because easyjson code generationinvokesgo run on a temporary file (an approach to code generation borrowedfromffjson).

someStruct:=&SomeStruct{Field1:"val1",Field2:"val2"}rawBytes,err:=easyjson.Marshal(someStruct)

someStruct:=&SomeStruct{}err:=easyjson.Unmarshal(rawBytes,someStruct)

Please see theGoDocfor more information and features.

Usage of easyjson:  -all    generate marshaler/unmarshalers for all structs in a file  -build_tags string        build tags to add to generated file  -gen_build_flags string        build flags when running the generator while bootstrapping  -byte        use simple bytes instead of Base64Bytes for slice of bytes  -leave_temps    do not delete temporary files  -no_std_marshalers    don't generate MarshalJSON/UnmarshalJSON funcs  -noformat    do not run 'gofmt -w' on output file  -omit_empty    omit empty fields by default  -output_filename string    specify the filename of the output  -pkg    process the whole package instead of just the given file  -snake_case    use snake_case names instead of CamelCase by default  -lower_camel_case        use lowerCamelCase instead of CamelCase by default  -stubs    only generate stubs for marshaler/unmarshaler funcs  -disallow_unknown_fields        return error if some unknown field in json appeared  -disable_members_unescape        disable unescaping of \uXXXX string sequences in member names

Using-all will generate marshalers/unmarshalers for all Go structs in thefile excluding those structs whose preceding comment starts witheasyjson:skip.For example:

//easyjson:skiptypeAstruct {}

If-all is not provided, then only those structs whose precedingcomment starts witheasyjson:json will have marshalers/unmarshalersgenerated. For example:

//easyjson:jsontypeAstruct {}

Additional option notes:

• -snake_case tells easyjson to generate snake_case field names by default(unless overridden by a field tag). The CamelCase to snake_case conversionalgorithm should work in most cases (ie, HTTPVersion will be converted to"http_version").

• -build_tags will add the specified build tags to generated Go sources.

• -gen_build_flags will execute the easyjson bootstapping code to launch theactual generator command with provided flags. Multiple arguments should beseparated by space e.g.-gen_build_flags="-mod=mod -x".

Besides standard json tag options like 'omitempty' the following are supported:

• 'nocopy' - disables allocation and copying of string values, making themrefer to original json buffer memory. This works great for short livedobjects which are not hold in memory after decoding and immediate usage.Note if string requires unescaping it will be processed as normally.
• 'intern' - string "interning" (deduplication) to save memory when the verysame string dictionary values are often met all over the structure.See below for more details.

For Go struct types, easyjson generates the funcsMarshalEasyJSON /UnmarshalEasyJSON for marshaling/unmarshaling JSON. In turn, these satisfytheeasyjson.Marshaler andeasyjson.Unmarshaler interfaces and when used inconjunction witheasyjson.Marshal /easyjson.Unmarshal avoid unnecessaryreflection / type assertions during marshaling/unmarshaling to/from JSON for Gostructs.

easyjson also generatesMarshalJSON andUnmarshalJSON funcs for Go structtypes compatible with the standardjson.Marshaler andjson.Unmarshalerinterfaces. Please be aware that using the standardjson.Marshal /json.Unmarshal for marshaling/unmarshaling will incur a significantperformance penalty when compared to usingeasyjson.Marshal /easyjson.Unmarshal.

Additionally, easyjson exposes utility funcs that use theMarshalEasyJSON andUnmarshalEasyJSON for marshaling/unmarshaling to and from standard readersand writers. For example, easyjson provideseasyjson.MarshalToHTTPResponseWriterwhich marshals to the standardhttp.ResponseWriter. Please see theGoDoclisting for the full listing ofutility funcs that are available.

Go types can provide their ownMarshalEasyJSON andUnmarshalEasyJSON funcsthat satisfy theeasyjson.Marshaler /easyjson.Unmarshaler interfaces.These will be used byeasyjson.Marshal andeasyjson.Unmarshal when definedfor a Go type.

Go types can also satisfy theeasyjson.Optional interface, which allows thetype to define its ownomitempty logic.

easyjson provides additional type wrappers defined in theeasyjson/optpackage. These wrap the standard Go primitives and in turn satisfy theeasyjson interfaces.

Theeasyjson/opt type wrappers are useful when needing to distinguish betweena missing value and/or when needing to specifying a default value. Typewrappers allow easyjson to avoid additional pointers and heap allocations andcan significantly increase performance when used properly.

easyjson uses a buffer pool that allocates data in increasing chunks from 128to 32768 bytes. Chunks of 512 bytes and larger will be reused with the help ofsync.Pool. The maximum size of a chunk is bounded to reduce redundant memoryallocation and to allow larger reusable buffers.

easyjson's custom allocation buffer pool is defined in theeasyjson/bufferpackage, and the default behavior pool behavior can be modified (if necessary)through a call tobuffer.Init() prior to any marshaling or unmarshaling.Please see theGoDoc listingfor more information.

During unmarshaling,string field values can be optionallyinterned to reduce memoryallocations and usage by deduplicating strings in memory, at the expense of slightlyincreased CPU usage.

This will work effectively only forstring fields being decoded that have frequentlythe same value (e.g. if you have a string field that can only assume a small numberof possible values).

To enable string interning, add theintern keyword tag to yourjson tag onstringfields, e.g.:

typeFoostruct {UUIDstring`json:"uuid"`// will not be interned during unmarshalingStatestring`json:"state,intern"`// will be interned during unmarshaling}

• easyjson is still early in its development. As such, there are likely to bebugs and missing features when compared toencoding/json. In the case of amissing feature or bug, please create a GitHub issue. Pull requests arewelcome!

• Unlikeencoding/json, object keys are case-sensitive. Case-insensitivematching is not currently provided due to the significant performance hitwhen doing case-insensitive key matching. In the future, case-insensitiveobject key matching may be provided via an option to the generator.

• easyjson makes use ofunsafe, which simplifies the code andprovides significant performance benefits by allowing no-copyconversion from[]byte tostring. That said,unsafe is usedonly when unmarshaling and parsing JSON, and anyunsafe operations/ memory allocations done will be safely deallocated byeasyjson. Set the build tageasyjson_nounsafe to compile itwithoutunsafe.

• easyjson is compatible with Google App Engine. Theappengine buildtag (set by App Engine's environment) will automatically disable theuse ofunsafe, which is not allowed in App Engine's StandardEnvironment. Note that the use with App Engine is still experimental.

• Floats are formatted using the default precision from Go'sstrconv package.As such, easyjson will not correctly handle high precision floats whenmarshaling/unmarshaling JSON. Note, however, that there are very few/limiteduses where this behavior is not sufficient for general use. That said, adifferent package may be needed if precise marshaling/unmarshaling of highprecision floats to/from JSON is required.

• While unmarshaling, the JSON parser does the minimal amount of work needed toskip over unmatching parens, and as such full validation is not done for theentire JSON value being unmarshaled/parsed.

• Currently there is no true streaming support for encoding/decoding astypically for many uses/protocols the final, marshaled length of the JSONneeds to be known prior to sending the data. Currently this is not possiblewith easyjson's architecture.

• easyjson parser and codegen based on reflection, so it won't work onpackage mainfiles, because they cant be imported by parser.

Most benchmarks were done using the example13kB example JSON(9k after eliminating whitespace). This example is similar to real-world data,is well-structured, and contains a healthy variety of different types, makingit ideal for JSON serialization benchmarks.

Note:

• For small request benchmarks, an 80 byte portion of the above example wasused.

• For large request marshaling benchmarks, a struct containing 50 regularsamples was used, making a ~500kB output JSON.

• Benchmarks are showing the results of easyjson's default behaviour,which makes use ofunsafe.

Benchmarks are available in the repository and can be run by invokingmake.

easyjson is roughly 5-6 times faster than the standardencoding/json forunmarshaling, and 3-4 times faster for non-concurrent marshaling. Concurrentmarshaling is 6-7x faster if marshaling to a writer.

easyjson uses the same approach for JSON marshaling asffjson, but takes a significantlydifferent approach to lexing and parsing JSON during unmarshaling. This meanseasyjson is roughly 2-3x faster for unmarshaling and 1.5-2x faster fornon-concurrent unmarshaling.

As of this writing,ffjson seems to have issues when used concurrently:specifically, large request pooling hurtsffjson's performance and causesscalability issues. These issues withffjson can likely be fixed, but as ofwriting remain outstanding/known issues withffjson.

easyjson andffjson have similar performance for small requests, howevereasyjson outperformsffjson by roughly 2-5x times for large requests whenused with a writer.

go/codec providescompile-time helpers for JSON generation. In this case, helpers do not worklike marshalers as they are encoding-independent.

easyjson is generally 2x faster thango/codec for non-concurrent benchmarksand about 3x faster for concurrent encoding (without marshaling to a writer).

In an attempt to measure marshaling performance ofgo/codec (as opposed toallocations/memcpy/writer interface invocations), a benchmark was done withresetting length of a byte slice rather than resetting the whole slice to nil.However, the optimization in this exact form may not be applicable in practice,since the memory is not freed between marshaling operations.

ujson is using C code for parsing, so itis interesting to see how plain golang compares to that. It is important to notethat the resulting object for python is slower to access, since the libraryparses JSON object into dictionaries.

easyjson is slightly faster for unmarshaling and 2-3x faster thanujson formarshaling.

ffjson results are from February 4th, 2016, using the latestffjson and go1.6.go/codec results are from March 4th, 2016, using the latestgo/codec and go1.6.

* marshaling to a writer,** usingffjson.Pool(),*** reusing output slice instead of resetting it to nil

* marshaling to a writer,** usingffjson.Pool(),*** reusing output slice instead of resetting it to nil