Repository files navigation

Goroutine local storage

It is my duty to point you tohttps://blog.golang.org/context, which is howGoogle solves all of the problems you'd perhaps consider using this packagefor at scale.

One downside to Google's approach is thatall of your functions must havea new first argument, but after clearing that hurdle everything else is muchbetter.

If you aren't interested in this warning, read on.

Every so often, a thread shows up on thegolang-nuts asking for someform of goroutine-local-storage, or some kind of goroutine id, or some kind ofcontext. There are a few valid use cases for goroutine-local-storage, one ofthe most prominent being log line context. One poster was interested in beingable to log an HTTP request context id in every log line in the same goroutineas the incoming HTTP request, without having to change every library andfunction call he was interested in logging.

This would be pretty useful. Provided that you could get some kind ofgoroutine-local-storage, you could calllog.SetOutput with your own loggingwriter that checks goroutine-local-storage for some context information andadds that context to your log lines.

But alas, Andrew Gerrand's typically diplomatic answer to the question ofgoroutine-local variables was:

We wouldn't even be having this discussion if thread local storage wasn'tuseful. But every feature comes at a cost, and in my opinion the cost ofthreadlocals far outweighs their benefits. They're just not a good fit forGo.

So, yeah, that makes sense. That's a pretty good reason for why the languagewon't support a specific and (relatively) unuseful feature that requires someruntime changes, just for the sake of a little bit of log improvement.

But does Go require runtime changes?

Go has pretty fantastic introspective and reflective features, but one thing Godoesn't give you is any kind of access to the stack pointer, or frame pointer,or goroutine id, or anything contextual about your current stack. It gives youaccess to your list of callers, but only along with program counters, which arefixed at compile time.

But it does give you the stack.

So, we define 16 special functions and embed base-16 tags into the stack usingthe call order of those 16 functions. Then, we can read our tags back out ofthe stack looking at the callers list.

We then use these tags as an index into a traditional map for implementingthis library.

"Wow, that's horrifying."

"This is the most terrible thing I have seen in a very long time."

"Where is it getting a context from? Is this serializing all the requests?What the heck is the client being bound to? What are these tags? Why does heneed callers? Oh god no. No no no."

Please see the docs athttp://godoc.org/github.com/jtolds/gls

If you're okay relying on the string format of the current runtime stacktraceincluding a unique goroutine id (not guaranteed by the spec or anything, butvery unlikely to change within a Go release), you might be able to squeezeout a bit more performance by using this similar library, inspired by somecode Brad Fitzpatrick wrote for debugging his HTTP/2 library:https://github.com/tylerb/gls (in contrast, jtolds/gls doesn't requireany knowledge of the string format of the runtime stacktrace, whichprobably adds unnecessary overhead).