- Notifications
You must be signed in to change notification settings - Fork3
go-faster/jx
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Package jx implements encoding and decoding of json [RFC 7159].Lightweight fork ofjsoniter.
go get github.com/go-faster/jx- Mostly zero-allocation and highly optimized
- Directly encode and decode json values
- No reflect or
interface{} - Pools and direct buffer access for less (or none) allocations
- Multi-pass decoding
- Validation
Seeusage for examples. Mostly suitable for fast low-level json manipulationwith high control, for dynamic parsing and encoding of unstructured data. Used inogen project forjson (un)marshaling code generation based on json and OpenAPI schemas.
For example, we have following OpenTelemetry log entry:
{"Timestamp":"1586960586000000000","Attributes": {"http.status_code":500,"http.url":"http://example.com","my.custom.application.tag":"hello" },"Resource": {"service.name":"donut_shop","service.version":"2.0.0","k8s.pod.uid":"1138528c-c36e-11e9-a1a7-42010a800198" },"TraceId":"13e2a0921288b3ff80df0a0482d4fc46","SpanId":"43222c2d51a7abe3","SeverityText":"INFO","SeverityNumber":9,"Body":"20200415T072306-0700 INFO I like donuts"}Flexibility ofjx enables highly efficient semantic-aware encoding and decoding,e.g. using[16]byte forTraceId with zero-allocationhex encoding in json:
| Name | Speed | Allocations |
|---|---|---|
| Decode | 1279 MB/s | 0 allocs/op |
| Validate | 1914 MB/s | 0 allocs/op |
| Encode | 1202 MB/s | 0 allocs/op |
| Write | 2055 MB/s | 0 allocs/op |
cpu: AMD Ryzen 9 7950X
Seeotel_test.go for example.
Most ofjsoniter issues are caused by necessityto be drop-in replacement for standardencoding/json. Removing such constrains greatlysimplified implementation and reduced scope, allowing to focus on json stream processing.
- Commas are handled automatically while encoding
- Raw json, Number and Base64 support
- Reduced scope
- No reflection
- No
encoding/jsonadapter - 3.5x less code (8.5K to 2.4K SLOC)
- Fuzzing, improved test coverage
- Drastically refactored and simplified
- Explicit error returns
- No
ConfigorAPI
Usejx.Decoder. Zero value is valid,but constructors are available for convenience:
- jx.Decode(reader io.Reader, bufSize int) for
io.Reader - jx.DecodeBytes([]byte) for byte slices
- jx.DecodeStr(string) for strings
To reuse decoders and their buffers, usejx.GetDecoderandjx.PutDecoder alongside with reset functions:
- jx.Decoder.Reset(io.Reader) to reset to new
io.Reader - jx.Decoder.ResetBytes([]byte) to decode another byte slice
Decoder is reset onPutDecoder.
d:=jx.DecodeStr(`{"values":[4,8,15,16,23,42]}`)// Save all integers from "values" array to slice.varvalues []int// Iterate over each object field.iferr:=d.Obj(func(d*jx.Decoder,keystring)error {switchkey {case"values":// Iterate over each array element.returnd.Arr(func(d*jx.Decoder)error {v,err:=d.Int()iferr!=nil {returnerr }values=append(values,v)returnnil })default:// Skip unknown fields if any.returnd.Skip() }});err!=nil {panic(err)}fmt.Println(values)// Output: [4 8 15 16 23 42]
Usejx.Encoder. Zero value is valid, reuse withjx.GetEncoder,jx.PutEncoder andjx.Encoder.Reset(). Encoder is reset onPutEncoder.
vare jx.Encodere.ObjStart()// {e.FieldStart("values")// "values":e.ArrStart()// [for_,v:=range []int{4,8,15,16,23,42} {e.Int(v)}e.ArrEnd()// ]e.ObjEnd()// }fmt.Println(e)fmt.Println("Buffer len:",len(e.Bytes()))// Output: {"values":[4,8,15,16,23,42]}// Buffer len: 28
Usejx.Writer for low level json writing.
No automatic commas or indentation for lowest possible overhead, useful for code generated json encoding.
Usejx.Decoder.Raw to read raw json values, similar tojson.RawMessage.
d:=jx.DecodeStr(`{"foo": [1, 2, 3]}`)varraw jx.Rawiferr:=d.Obj(func(d*jx.Decoder,keystring)error {v,err:=d.Raw()iferr!=nil {returnerr }raw=vreturnnil});err!=nil {panic(err)}fmt.Println(raw.Type(),raw)// Output:// array [1, 2, 3]
Usejx.Decoder.Num to read numbers, similar tojson.Number.Also supports number strings, like"12345", which is common compatible way to representuint64.
d:=jx.DecodeStr(`{"foo": "10531.0"}`)varn jx.Numiferr:=d.Obj(func(d*jx.Decoder,keystring)error {v,err:=d.Num()iferr!=nil {returnerr }n=vreturnnil});err!=nil {panic(err)}fmt.Println(n)fmt.Println("positive:",n.Positive())// Can decode floats with zero fractional part as integers:v,err:=n.Int64()iferr!=nil {panic(err)}fmt.Println("int64:",v)// Output:// "10531.0"// positive: true// int64: 10531
Usejx.Encoder.Base64 andjx.Decoder.Base64 orjx.Decoder.Base64Append.
Same as encoding/json, base64.StdEncoding or [RFC 4648].
vare jx.Encodere.Base64([]byte("Hello"))fmt.Println(e)data,_:=jx.DecodeBytes(e.Bytes()).Base64()fmt.Printf("%s",data)// Output:// "SGVsbG8="// Hello
Check that byte slice is valid json withjx.Valid:
fmt.Println(jx.Valid([]byte(`{"field": "value"}`)))// truefmt.Println(jx.Valid([]byte(`"Hello, world!"`)))// truefmt.Println(jx.Valid([]byte(`["foo"}`)))// false
Thejx.Decoder.Capture method allows to unread everything is read in callback.Useful for multi-pass parsing:
d:=jx.DecodeStr(`["foo", "bar", "baz"]`)varelemsint// NB: Currently Capture does not support io.Reader, only buffers.iferr:=d.Capture(func(d*jx.Decoder)error {// Everything decoded in this callback will be rolled back.returnd.Arr(func(d*jx.Decoder)error {elems++returnd.Skip()})});err!=nil {panic(err)}// Decoder is rolled back to state before "Capture" call.fmt.Println("Read",elems,"elements on first pass")fmt.Println("Next element is",d.Next(),"again")// Output:// Read 3 elements on first pass// Next element is array again
TheDecoder.ObjBytes method tries not to allocate memory for keys, reusing existing buffer.
d:=DecodeStr(`{"id":1,"randomNumber":10}`)d.ObjBytes(func(d*Decoder,key []byte)error {switchstring(key) {case"id":case"randomNumber": }returnd.Skip()})
- Rework and export
Any - Support
Rawfor io.Reader - Support
Capturefor io.Reader - Improve Num
- Better validation on decoding
- Support BigFloat and BigInt
- Support equivalence check, like
eq(1.0, 1) == true
- Add non-callback decoding of objects
- Code generation for decoding or encoding
- Replacement for
encoding/json - Reflection or
interface{}based encoding or decoding - Support for json path or similar
This package should be kept as simple as possible and be used aslow-level foundation for high-level projects like code generator.
MIT, same as jsoniter
About
json encoding and decoding
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Uh oh!
There was an error while loading.Please reload this page.