- Notifications
You must be signed in to change notification settings - Fork587
Zero Allocation JSON Logger
License
rs/zerolog
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
The zerolog package provides a fast and simple logger dedicated to JSON output.
Zerolog's API is designed to provide both a great developer experience and stunningperformance. Its unique chaining API allows zerolog to write JSON (or CBOR) log events by avoiding allocations and reflection.
Uber'szap library pioneered this approach. Zerolog is taking this concept to the next level with a simpler to use API and even better performance.
To keep the code base and the API simple, zerolog focuses on efficient structured logging only. Pretty logging on the console is made possible using the provided (but inefficient)zerolog.ConsoleWriter
.
Find outwho uses zerolog and add your company / project to the list.
- Blazing fast
- Low to zero allocation
- Leveled logging
- Sampling
- Hooks
- Contextual fields
context.Context
integration- Integration with
net/http
- JSON and CBOR encoding formats
- Pretty logging for development
- Error Logging (with optional Stacktrace)
go get -u github.com/rs/zerolog/log
For simple logging, import the global logger packagegithub.com/rs/zerolog/log
package mainimport ("github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {// UNIX Time is faster and smaller than most timestampszerolog.TimeFieldFormat=zerolog.TimeFormatUnixlog.Print("hello world")}// Output: {"time":1516134303,"level":"debug","message":"hello world"}
Note: By default log writes to
os.Stderr
Note: The default log level forlog.Print
istrace
zerolog allows data to be added to log messages in the form of key:value pairs. The data added to the message adds "context" about the log event that can be critical for debugging as well as myriad other purposes. An example of this is below:
package mainimport ("github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {zerolog.TimeFieldFormat=zerolog.TimeFormatUnixlog.Debug().Str("Scale","833 cents").Float64("Interval",833.09).Msg("Fibonacci is everywhere")log.Debug().Str("Name","Tom").Send()}// Output: {"level":"debug","Scale":"833 cents","Interval":833.09,"time":1562212768,"message":"Fibonacci is everywhere"}// Output: {"level":"debug","Name":"Tom","time":1562212768}
You'll note in the above example that when adding contextual fields, the fields are strongly typed. You can find the full list of supported fieldshere
package mainimport ("github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {zerolog.TimeFieldFormat=zerolog.TimeFormatUnixlog.Info().Msg("hello world")}// Output: {"time":1516134303,"level":"info","message":"hello world"}
It is very important to note that when using thezerolog chaining API, as shown above (
log.Info().Msg("hello world"
), the chain must have either theMsg
orMsgf
method call. If you forget to add either of these, the log will not occur and there is no compile time error to alert you of this.
zerolog allows for logging at the following levels (from highest to lowest):
- panic (
zerolog.PanicLevel
, 5) - fatal (
zerolog.FatalLevel
, 4) - error (
zerolog.ErrorLevel
, 3) - warn (
zerolog.WarnLevel
, 2) - info (
zerolog.InfoLevel
, 1) - debug (
zerolog.DebugLevel
, 0) - trace (
zerolog.TraceLevel
, -1)
You can set the Global logging level to any of these options using theSetGlobalLevel
function in the zerolog package, passing in one of the given constants above, e.g.zerolog.InfoLevel
would be the "info" level. Whichever level is chosen, all logs with a level greater than or equal to that level will be written. To turn off logging entirely, pass thezerolog.Disabled
constant.
This example uses command-line flags to demonstrate various outputs depending on the chosen log level.
package mainimport ("flag""github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {zerolog.TimeFieldFormat=zerolog.TimeFormatUnixdebug:=flag.Bool("debug",false,"sets log level to debug")flag.Parse()// Default level for this example is info, unless debug flag is presentzerolog.SetGlobalLevel(zerolog.InfoLevel)if*debug {zerolog.SetGlobalLevel(zerolog.DebugLevel) }log.Debug().Msg("This message appears only when log level set to Debug")log.Info().Msg("This message appears when log level set to Debug or Info")ife:=log.Debug();e.Enabled() {// Compute log output only if enabled.value:="bar"e.Str("foo",value).Msg("some debug message") }}
Info Output (no flag)
$ ./logLevelExample{"time":1516387492,"level":"info","message":"This message appears when log level set to Debug or Info"}
Debug Output (debug flag set)
$ ./logLevelExample -debug{"time":1516387573,"level":"debug","message":"This message appears only when log level set to Debug"}{"time":1516387573,"level":"info","message":"This message appears when log level set to Debug or Info"}{"time":1516387573,"level":"debug","foo":"bar","message":"some debug message"}
You may choose to log without a specific level by using theLog
method. You may also write without a message by setting an empty string in themsg string
parameter of theMsg
method. Both are demonstrated in the example below.
package mainimport ("github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {zerolog.TimeFieldFormat=zerolog.TimeFormatUnixlog.Log().Str("foo","bar").Msg("")}// Output: {"time":1494567715,"foo":"bar"}
You can log errors using theErr
method
package mainimport ("errors""github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {zerolog.TimeFieldFormat=zerolog.TimeFormatUnixerr:=errors.New("seems we have an error here")log.Error().Err(err).Msg("")}// Output: {"level":"error","error":"seems we have an error here","time":1609085256}
The default field name for errors is
error
, you can change this by settingzerolog.ErrorFieldName
to meet your needs.
Usinggithub.com/pkg/errors
, you can add a formatted stacktrace to your errors.
package mainimport ("github.com/pkg/errors""github.com/rs/zerolog/pkgerrors""github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {zerolog.TimeFieldFormat=zerolog.TimeFormatUnixzerolog.ErrorStackMarshaler=pkgerrors.MarshalStackerr:=outer()log.Error().Stack().Err(err).Msg("")}funcinner()error {returnerrors.New("seems we have an error here")}funcmiddle()error {err:=inner()iferr!=nil {returnerr}returnnil}funcouter()error {err:=middle()iferr!=nil {returnerr}returnnil}// Output: {"level":"error","stack":[{"func":"inner","line":"20","source":"errors.go"},{"func":"middle","line":"24","source":"errors.go"},{"func":"outer","line":"32","source":"errors.go"},{"func":"main","line":"15","source":"errors.go"},{"func":"main","line":"204","source":"proc.go"},{"func":"goexit","line":"1374","source":"asm_amd64.s"}],"error":"seems we have an error here","time":1609086683}
zerolog.ErrorStackMarshaler must be set in order for the stack to output anything.
package mainimport ("errors""github.com/rs/zerolog""github.com/rs/zerolog/log")funcmain() {err:=errors.New("A repo man spends his life getting into tense situations")service:="myservice"zerolog.TimeFieldFormat=zerolog.TimeFormatUnixlog.Fatal().Err(err).Str("service",service).Msgf("Cannot start %s",service)}// Output: {"time":1516133263,"level":"fatal","error":"A repo man spends his life getting into tense situations","service":"myservice","message":"Cannot start myservice"}// exit status 1
NOTE: Using
Msgf
generates one allocation even when the logger is disabled.
logger:=zerolog.New(os.Stderr).With().Timestamp().Logger()logger.Info().Str("foo","bar").Msg("hello world")// Output: {"level":"info","time":1494567715,"message":"hello world","foo":"bar"}
sublogger:=log.With().Str("component","foo").Logger()sublogger.Info().Msg("hello world")// Output: {"level":"info","time":1494567715,"message":"hello world","component":"foo"}
To log a human-friendly, colorized output, usezerolog.ConsoleWriter
:
log.Logger=log.Output(zerolog.ConsoleWriter{Out:os.Stderr})log.Info().Str("foo","bar").Msg("Hello world")// Output: 3:04PM INF Hello World foo=bar
To customize the configuration and formatting:
output:= zerolog.ConsoleWriter{Out:os.Stdout,TimeFormat:time.RFC3339}output.FormatLevel=func(iinterface{})string {returnstrings.ToUpper(fmt.Sprintf("| %-6s|",i))}output.FormatMessage=func(iinterface{})string {returnfmt.Sprintf("***%s****",i)}output.FormatFieldName=func(iinterface{})string {returnfmt.Sprintf("%s:",i)}output.FormatFieldValue=func(iinterface{})string {returnstrings.ToUpper(fmt.Sprintf("%s",i))}log:=zerolog.New(output).With().Timestamp().Logger()log.Info().Str("foo","bar").Msg("Hello World")// Output: 2006-01-02T15:04:05Z07:00 | INFO | ***Hello World**** foo:BAR
To use custom advanced formatting:
output:= zerolog.ConsoleWriter{Out:os.Stdout,NoColor:true,PartsOrder: []string{"level","one","two","three","message"},FieldsExclude: []string{"one","two","three"}}output.FormatLevel=func(iinterface{})string {returnstrings.ToUpper(fmt.Sprintf("%-6s",i)) }output.FormatFieldName=func(iinterface{})string {returnfmt.Sprintf("%s:",i) }output.FormatPartValueByName=func(iinterface{},sstring)string {varretstringswitchs {case"one":ret=strings.ToUpper(fmt.Sprintf("%s",i))case"two":ret=strings.ToLower(fmt.Sprintf("%s",i))case"three":ret=strings.ToLower(fmt.Sprintf("(%s)",i)) }returnret}log:=zerolog.New(output)log.Info().Str("foo","bar").Str("two","TEST_TWO").Str("one","test_one").Str("three","test_three").Msg("Hello World")// Output: INFO TEST_ONE test_two (test_three) Hello World foo:bar
log.Info().Str("foo","bar").Dict("dict",zerolog.Dict().Str("bar","baz").Int("n",1), ).Msg("hello world")// Output: {"level":"info","time":1494567715,"foo":"bar","dict":{"bar":"baz","n":1},"message":"hello world"}
zerolog.TimestampFieldName="t"zerolog.LevelFieldName="l"zerolog.MessageFieldName="m"log.Info().Msg("hello world")// Output: {"l":"info","t":1494567715,"m":"hello world"}
log.Logger=log.With().Str("foo","bar").Logger()
Equivalent ofLlongfile
:
log.Logger=log.With().Caller().Logger()log.Info().Msg("hello world")// Output: {"level": "info", "message": "hello world", "caller": "/go/src/your_project/some_file:21"}
Equivalent ofLshortfile
:
zerolog.CallerMarshalFunc=func(pcuintptr,filestring,lineint)string {returnfilepath.Base(file)+":"+strconv.Itoa(line)}log.Logger=log.With().Caller().Logger()log.Info().Msg("hello world")// Output: {"level": "info", "message": "hello world", "caller": "some_file:21"}
If your writer might be slow or not thread-safe and you need your log producers to never get slowed down by a slow writer, you can use adiode.Writer
as follows:
wr:=diode.NewWriter(os.Stdout,1000,10*time.Millisecond,func(missedint) {fmt.Printf("Logger Dropped %d messages",missed)})log:=zerolog.New(wr)log.Print("test")
You will need to installcode.cloudfoundry.org/go-diodes
to use this feature.
sampled:=log.Sample(&zerolog.BasicSampler{N:10})sampled.Info().Msg("will be logged every 10 messages")// Output: {"time":1494567715,"level":"info","message":"will be logged every 10 messages"}
More advanced sampling:
// Will let 5 debug messages per period of 1 second.// Over 5 debug message, 1 every 100 debug messages are logged.// Other levels are not sampled.sampled:=log.Sample(zerolog.LevelSampler{DebugSampler:&zerolog.BurstSampler{Burst:5,Period:1*time.Second,NextSampler:&zerolog.BasicSampler{N:100}, },})sampled.Debug().Msg("hello world")// Output: {"time":1494567715,"level":"debug","message":"hello world"}
typeSeverityHookstruct{}func (hSeverityHook)Run(e*zerolog.Event,level zerolog.Level,msgstring) {iflevel!=zerolog.NoLevel {e.Str("severity",level.String()) }}hooked:=log.Hook(SeverityHook{})hooked.Warn().Msg("")// Output: {"level":"warn","severity":"warn"}
ctx:=log.With().Str("component","module").Logger().WithContext(ctx)log.Ctx(ctx).Info().Msg("hello world")// Output: {"component":"module","level":"info","message":"hello world"}
log:=zerolog.New(os.Stdout).With().Str("foo","bar").Logger()stdlog.SetFlags(0)stdlog.SetOutput(log)stdlog.Print("hello world")// Output: {"foo":"bar","message":"hello world"}
Go contexts are commonly passed throughout Go code, and this can help you passyour Logger into places it might otherwise be hard to inject. TheLogger
instance may be attached to Go context (context.Context
) usingLogger.WithContext(ctx)
and extracted from it usingzerolog.Ctx(ctx)
.For example:
funcf() {logger:=zerolog.New(os.Stdout)ctx:=context.Background()// Attach the Logger to the context.Contextctx=logger.WithContext(ctx)someFunc(ctx)}funcsomeFunc(ctx context.Context) {// Get Logger from the go Context. if it's nil, then// `zerolog.DefaultContextLogger` is returned, if// `DefaultContextLogger` is nil, then a disabled logger is returned.logger:=zerolog.Ctx(ctx)logger.Info().Msg("Hello")}
A second form ofcontext.Context
integration allows you to pass the currentcontext.Context into the logged event, and retrieve it from hooks. This can beuseful to log trace and span IDs or other information stored in the go context,and facilitates the unification of logging and tracing in some systems:
typeTracingHookstruct{}func (hTracingHook)Run(e*zerolog.Event,level zerolog.Level,msgstring) {ctx:=e.GetCtx()spanId:=getSpanIdFromContext(ctx)// as per your tracing frameworke.Str("span-id",spanId)}funcf() {// Setup the loggerlogger:=zerolog.New(os.Stdout)logger=logger.Hook(TracingHook{})ctx:=context.Background()// Use the Ctx function to make the context available to the hooklogger.Info().Ctx(ctx).Msg("Hello")}
Thegithub.com/rs/zerolog/hlog
package provides some helpers to integrate zerolog withhttp.Handler
.
In this example we usealice to install logger for better readability.
log:=zerolog.New(os.Stdout).With().Timestamp().Str("role","my-service").Str("host",host).Logger()c:=alice.New()// Install the logger handler with default output on the consolec=c.Append(hlog.NewHandler(log))// Install some provided extra handler to set some request's context fields.// Thanks to that handler, all our logs will come with some prepopulated fields.c=c.Append(hlog.AccessHandler(func(r*http.Request,status,sizeint,duration time.Duration) {hlog.FromRequest(r).Info().Str("method",r.Method).Stringer("url",r.URL).Int("status",status).Int("size",size).Dur("duration",duration).Msg("")}))c=c.Append(hlog.RemoteAddrHandler("ip"))c=c.Append(hlog.UserAgentHandler("user_agent"))c=c.Append(hlog.RefererHandler("referer"))c=c.Append(hlog.RequestIDHandler("req_id","Request-Id"))// Here is your final handlerh:=c.Then(http.HandlerFunc(func(w http.ResponseWriter,r*http.Request) {// Get the logger from the request's context. You can safely assume it// will be always there: if the handler is removed, hlog.FromRequest// will return a no-op logger.hlog.FromRequest(r).Info().Str("user","current user").Str("status","ok").Msg("Something happened")// Output: {"level":"info","time":"2001-02-03T04:05:06Z","role":"my-service","host":"local-hostname","req_id":"b4g0l5t6tfid6dtrapu0","user":"current user","status":"ok","message":"Something happened"}}))http.Handle("/",h)iferr:=http.ListenAndServe(":8080",nil);err!=nil {log.Fatal().Err(err).Msg("Startup failed")}
zerolog.MultiLevelWriter
may be used to send the log message to multiple outputs.In this example, we send the log message to bothos.Stdout
and the in-built ConsoleWriter.
funcmain() {consoleWriter:= zerolog.ConsoleWriter{Out:os.Stdout}multi:=zerolog.MultiLevelWriter(consoleWriter,os.Stdout)logger:=zerolog.New(multi).With().Timestamp().Logger()logger.Info().Msg("Hello World!")}// Output (Line 1: Console; Line 2: Stdout)// 12:36PM INF Hello World!// {"level":"info","time":"2019-11-07T12:36:38+03:00","message":"Hello World!"}
Some settings can be changed and will be applied to all loggers:
log.Logger
: You can set this value to customize the global logger (the one used by package level methods).zerolog.SetGlobalLevel
: Can raise the minimum level of all loggers. Call this withzerolog.Disabled
to disable logging altogether (quiet mode).zerolog.DisableSampling
: If argument istrue
, all sampled loggers will stop sampling and issue 100% of their log events.zerolog.TimestampFieldName
: Can be set to customizeTimestamp
field name.zerolog.LevelFieldName
: Can be set to customize level field name.zerolog.MessageFieldName
: Can be set to customize message field name.zerolog.ErrorFieldName
: Can be set to customizeErr
field name.zerolog.TimeFieldFormat
: Can be set to customizeTime
field value formatting. If set withzerolog.TimeFormatUnix
,zerolog.TimeFormatUnixMs
orzerolog.TimeFormatUnixMicro
, times are formatted as UNIX timestamp.zerolog.DurationFieldUnit
: Can be set to customize the unit for time.Duration type fields added byDur
(default:time.Millisecond
).zerolog.DurationFieldInteger
: If set totrue
,Dur
fields are formatted as integers instead of floats (default:false
).zerolog.ErrorHandler
: Called whenever zerolog fails to write an event on its output. If not set, an error is printed on the stderr. This handler must be thread safe and non-blocking.zerolog.FloatingPointPrecision
: If set to a value other than -1, controls the numberof digits when formatting float numbers in JSON. Seestrconv.FormatFloatfor more details.
Str
Bool
Int
,Int8
,Int16
,Int32
,Int64
Uint
,Uint8
,Uint16
,Uint32
,Uint64
Float32
,Float64
Err
: Takes anerror
and renders it as a string using thezerolog.ErrorFieldName
field name.Func
: Run afunc
only if the level is enabled.Timestamp
: Inserts a timestamp field withzerolog.TimestampFieldName
field name, formatted usingzerolog.TimeFieldFormat
.Time
: Adds a field with time formatted withzerolog.TimeFieldFormat
.Dur
: Adds a field withtime.Duration
.Dict
: Adds a sub-key/value as a field of the event.RawJSON
: Adds a field with an already encoded JSON ([]byte
)Hex
: Adds a field with value formatted as a hexadecimal string ([]byte
)Interface
: Uses reflection to marshal the type.
Most fields are also available in the slice format (Strs
for[]string
,Errs
for[]error
etc.)
In addition to the default JSON encoding,zerolog
can produce binary logs usingCBOR encoding. The choice of encoding can be decided at compile time using the build tagbinary_log
as follows:
go build -tags binary_log.
To Decode binary encoded log files you can use any CBOR decoder. One has been tested to workwith zerolog library isCSD.
- grpc-zerolog: Implementation of
grpclog.LoggerV2
interface usingzerolog
- overlog: Implementation of
Mapped Diagnostic Context
interface usingzerolog
- zerologr: Implementation of
logr.LogSink
interface usingzerolog
Seelogbench for more comprehensive and up-to-date benchmarks.
All operations are allocation free (those numbersinclude JSON encoding):
BenchmarkLogEmpty-8 100000000 19.1 ns/op 0 B/op 0 allocs/opBenchmarkDisabled-8 500000000 4.07 ns/op 0 B/op 0 allocs/opBenchmarkInfo-8 30000000 42.5 ns/op 0 B/op 0 allocs/opBenchmarkContextFields-8 30000000 44.9 ns/op 0 B/op 0 allocs/opBenchmarkLogFields-8 10000000 184 ns/op 0 B/op 0 allocs/op
There are a few Go logging benchmarks and comparisons that include zerolog.
Using Uber's zap comparison benchmark:
Log a message and 10 fields:
Library | Time | Bytes Allocated | Objects Allocated |
---|---|---|---|
zerolog | 767 ns/op | 552 B/op | 6 allocs/op |
⚡ zap | 848 ns/op | 704 B/op | 2 allocs/op |
⚡ zap (sugared) | 1363 ns/op | 1610 B/op | 20 allocs/op |
go-kit | 3614 ns/op | 2895 B/op | 66 allocs/op |
lion | 5392 ns/op | 5807 B/op | 63 allocs/op |
logrus | 5661 ns/op | 6092 B/op | 78 allocs/op |
apex/log | 15332 ns/op | 3832 B/op | 65 allocs/op |
log15 | 20657 ns/op | 5632 B/op | 93 allocs/op |
Log a message with a logger that already has 10 fields of context:
Library | Time | Bytes Allocated | Objects Allocated |
---|---|---|---|
zerolog | 52 ns/op | 0 B/op | 0 allocs/op |
⚡ zap | 283 ns/op | 0 B/op | 0 allocs/op |
⚡ zap (sugared) | 337 ns/op | 80 B/op | 2 allocs/op |
lion | 2702 ns/op | 4074 B/op | 38 allocs/op |
go-kit | 3378 ns/op | 3046 B/op | 52 allocs/op |
logrus | 4309 ns/op | 4564 B/op | 63 allocs/op |
apex/log | 13456 ns/op | 2898 B/op | 51 allocs/op |
log15 | 14179 ns/op | 2642 B/op | 44 allocs/op |
Log a static string, without any context orprintf
-style templating:
Library | Time | Bytes Allocated | Objects Allocated |
---|---|---|---|
zerolog | 50 ns/op | 0 B/op | 0 allocs/op |
⚡ zap | 236 ns/op | 0 B/op | 0 allocs/op |
standard library | 453 ns/op | 80 B/op | 2 allocs/op |
⚡ zap (sugared) | 337 ns/op | 80 B/op | 2 allocs/op |
go-kit | 508 ns/op | 656 B/op | 13 allocs/op |
lion | 771 ns/op | 1224 B/op | 10 allocs/op |
logrus | 1244 ns/op | 1505 B/op | 27 allocs/op |
apex/log | 2751 ns/op | 584 B/op | 11 allocs/op |
log15 | 5181 ns/op | 1592 B/op | 26 allocs/op |
Note that zerolog does no de-duplication of fields. Using the same key multiple times creates multiple keys in final JSON:
logger:=zerolog.New(os.Stderr).With().Timestamp().Logger()logger.Info().Timestamp().Msg("dup")// Output: {"level":"info","time":1494567715,"time":1494567715,"message":"dup"}
In this case, many consumers will take the last value, but this is not guaranteed; check yours if in doubt.
Be careful when calling UpdateContext. It is not concurrency safe. Use the With method to create a child logger:
funchandler(w http.ResponseWriter,r*http.Request) {// Create a child logger for concurrency safetylogger:=log.Logger.With().Logger()// Add context fields, for example User-Agent from HTTP headerslogger.UpdateContext(func(c zerolog.Context) zerolog.Context {... })}
About
Zero Allocation JSON Logger