- Notifications
You must be signed in to change notification settings - Fork11
10x faster dynamic Protobuf parsing in Go that’s even 3x faster than generated code.
License
bufbuild/hyperpb-go
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
hyperpb is a highly optimized dynamic message library for Protobuf or read-onlyworkloads. It is designed to be a drop-in replacement fordynamicpb,protobuf-go's canonical solution for working with completely dynamic messages.
hyperpb's parser is an efficient VM for a special instruction set, a variant oftable-driven parsing (TDP), pioneered bythe UPB project.
Our parser is very fast, beatingdynamicpb by 10x, and often beatingprotobuf-go's generated code by a factor of 2-3x, especially for workloads withmany nested messages.
Here, we show two benchmark variants forhyperpb: out-of-the-box performance with no optimizations turned on, and real-time profile-guided optimization (PGO) with all optimizations we currently offer enabled. These benchmarks are a subset of the ones you can runusingmake bench.
You can read abouthyperpb on the Buf blog, or on project leadMiguel's blog.
The core conceit ofhyperpb is that you must pre-compile a parser usinghyperpb.Compile at runtime, much like regular expressions require that youregexp.Compile them. Doing this allowshyperpb to run optimization passes onyour message, and delaying it to runtime allows us to continuously improvelayout optimizations, without making any source-breaking changes.
For example, let's say that we want to compile a parser for some type baked intoour binary, and parse some data with it.
package mainimport ("fmt""log""buf.build/go/hyperpb""google.golang.org/protobuf/proto" weatherv1"buf.build/gen/go/bufbuild/hyperpb-examples/protocolbuffers/go/example/weather/v1")// Byte slice representation of a valid *weatherv1.WeatherReport.varweatherDataBytes= []byte{0x0a,0x07,0x53,0x65,0x61,0x74,0x74,0x6c,0x65,0x12,0x1d,0x0a,0x05,0x4b,0x41,0x44,0x39,0x33,0x15,0x66,0x86,0x22,0x43,0x1d,0xcd,0xcc,0x34,0x41,0x25,0xd7,0xa3,0xf0,0x41,0x2d,0x33,0x33,0x13,0x40,0x30,0x03,0x12,0x1d,0x0a,0x05,0x4b,0x48,0x42,0x36,0x30,0x15,0xcd,0x8c,0x22,0x43,0x1d,0x33,0x33,0x5b,0x41,0x25,0x52,0xb8,0xe0,0x41,0x2d,0x33,0x33,0xf3,0x3f,0x30,0x03,}funcmain() {// Compile a type for your message. Make sure to cache this!// Here, we're using a compiled-in descriptor.msgType:=hyperpb.CompileMessageDescriptor( (*weatherv1.WeatherReport)(nil).ProtoReflect().Descriptor(), )// Allocate a fresh message using that type.msg:=hyperpb.NewMessage(msgType)// Parse the message, using proto.Unmarshal like any other message type.iferr:=proto.Unmarshal(weatherDataBytes,msg);err!=nil {// Handle parse failure.log.Fatalf("failed to parse weather data: %v",err) }// Use reflection to read some fields. hyperpb currently only supports access// by reflection. You can also look up fields by index using fields.Get(), which// is less legible but doesn't hit a hashmap.fields:=msgType.Descriptor().Fields()// Get returns a protoreflect.Value, which can be printed directly...fmt.Println(msg.Get(fields.ByName("region")))// ... or converted to an explicit type to operate on, such as with List(),// which converts a repeated field into something with indexing operations.stations:=msg.Get(fields.ByName("weather_stations")).List()fori:=rangestations.Len() {// Get returns a protoreflect.Value too, so we need to convert it into// a message to keep extracting fields.station:=stations.Get(i).Message()fields:=station.Descriptor().Fields()// Here we extract each of the fields we care about from the message.// Again, we could use fields.Get if we know the indices.fmt.Println("station:",station.Get(fields.ByName("station")))fmt.Println("frequency:",station.Get(fields.ByName("frequency")))fmt.Println("temperature:",station.Get(fields.ByName("temperature")))fmt.Println("pressure:",station.Get(fields.ByName("pressure")))fmt.Println("wind_speed:",station.Get(fields.ByName("wind_speed")))fmt.Println("conditions:",station.Get(fields.ByName("conditions"))) }}
Currently,hyperpb only supports manipulating messages through the reflectionAPI; it shines best when you need write a very generic service thatdownloads types off the network and parses messages using those types, whichforces you to use reflection.
Mutation is currently not supported; any operation which would mutate analready-parsed message will panic. Which methods of*hyperpb.Message panicis included in the documentation.
We can use thehyperpb.CompileFileDescriptorSet function to parse a dynamic type anduse it to walk the fields of a message:
funcprocessDynamicMessage(schema*descriptorpb.FileDescriptorSet,messageName protoreflect.FullName,data []byte,)error {msgType,err:=hyperpb.CompileFileDescriptorSet(schema,messageName)// Remember to cache this!iferr!=nil {returnerr }msg:=hyperpb.NewMessage(msgType)iferr:=proto.Unmarshal(data,msg);err!=nil {returnerr }// Range will iterate over all of the populated fields in msg. Here we// use Range with go1.24 iterator syntax.forfield,value:=rangemsg.Range {// Do something with each populated field. }returnnil}
Since any generic, non-mutating operation will work withhyperpb messages,we can use them as an efficient transcoding medium from the wire format, forruntime-loaded messages.
funcdynamicMessageToJSON(schema*descriptorpb.FileDescriptorSet,messageName protoreflect.FullName,data []byte,) ([]byte,error) {msgType,err:=hyperpb.CompileFileDescriptorSet(schema,messageName)iferr!=nil {returnnil,err }msg:=hyperpb.NewMessage(msgType)iferr:=proto.Unmarshal(data,msg);err!=nil {returnnil,err }// Dump the message to JSON. This just works!returnprotojson.Marshal(msg)}
protovalidate also works directly on reflection, so it works out-of-the-box:
funcvalidateDynamicMessage(schema*descriptorpb.FileDescriptorSet,messageName protoreflect.FullName,data []byte,)error {// Unmarshal like before.msgType,err:=hyperpb.CompileFileDescriptorSet(schema,messageName)iferr!=nil {returnerr }msg:=hyperpb.NewMessage(msgType)iferr:=proto.Unmarshal(data,msg);err!=nil {returnerr }// Run custom validation. This just works!returnprotovalidate.Validate(msg)}
hyperpb is all about parsing as fast as possible, so there are a number ofoptimization knobs available. CallingMessage.Unmarshal directly insteadofproto.Unmarshal allows setting customUnmarshalOptions:
funcunmarshalWithCustomOptions(schema*descriptorpb.FileDescriptorSet,messageName protoreflect.FullName,data []byte,)error {msgType,err:=hyperpb.CompileFileDescriptorSet(schema,messageName)iferr!=nil {returnerr }msg:=hyperpb.NewMessage(msgType)returnmsg.Unmarshal(data,hyperpb.WithMaxDecodeMisses(16),// Additional options... )}
The compiler also takesCompileOptions, such as for configuring how extensionsare resolved:
msgType,err:=hyperpb.CompileFileDescriptor(schema,messageName,hyperpb.WithExtensionsFromTypes(typeRegistry),// Additional options...)
hyperpb also has a memory-reuse mechanism that side-steps the Go garbagecollector for improved allocation latency.hyperpb.Shared is book-keepingstate and resources shared by all messages resulting from the same parse.After the message goes out of scope, these resources are ordinarily reclaimedby the garbage collector.
However, ahyperpb.Shared can be retained after its associated message goesaway, allowing for re-use. Consider the following example of a request handler:
typerequestContextstruct {shared*hyperpb.Sharedtypesmap[string]*hyperpb.MessageType// Additional context fields...}func (c*requestContext)Handle(reqRequest) {msgType:=c.types[req.Type]msg:=c.shared.NewMessage(msgType)deferc.shared.Free()c.process(msg,req,...)}
Beware thatmsg must not outlive the call toShared.Free; failure to do sowill result in memory errors that Go cannot protect you from.
hyperpb supports online PGO for squeezing extra performance out of the parserby optimizing the parser with knowledge of what the average message actuallylooks like. For example, using PGO, the parser can predict the expected size ofrepeated fields and allocate more intelligently.
For example, suppose you have a corpus of messages for a particular type. Youcan build an optimized type, using that corpus as the profile, usingType.Recompile:
funccompilePGO(md protoreflect.MessageDescriptor,corpus [][]byte,) (*hyperpb.MessageType,error) {// Compile the type without any profiling information.msgType:=hyperpb.CompileMessageDescriptor(md)// Construct a new profile recorder.profile:=msgType.NewProfile()// Parse all of the specimens in the corpus, making sure to record a profile// for all of them.s:=new(hyperpb.Shared)for_,specimen:=rangecorpus {iferr:=s.NewMessage(msgType).Unmarshal(specimen,hyperpb.WithRecordProfile(profile,1.0), );err!=nil {returnnil,err }s.Free() }// Recompile with the profile.returnmsgType.Recompile(profile),nil}
Using a custom sampling rate inhyperpb.WithRecordProfile, it's possible tosample data on-line as part of a request flow, and recompile dynamically:
typerequestContextstruct {shared*hyperpb.Sharedtypesmap[string]*typeInfo// Additional context fields...}typetypeInfostruct {msgType atomic.Pointer[hyperpb.MessageType]prof atomic.Pointer[hyperpb.Profile]seen atomic.Int64}func (c*requestContext)Handle(reqRequest) {// Look up the type in the context's type map.typeInfo:=c.types[req.Type]// Parse the type as usual.msg:=c.shared.NewMessage(typeInfo.msgType.Load())deferc.shared.Free()iferr:=msg.Unmarshal(data,// Only profile 1% of messages.hyperpb.WithRecordProfile(typeInfo.prof.Load(),0.01), );err!=nil {// Process error... }typeInfo.seen.Add(1)// Every 100,000 messages, spawn a goroutine to asynchronously recompile the type.iftypeInfo.seen.Load()%100000==0 {gofunc() {prof:=typeInfo.prof.Load()if!typeInfo.prof.CompareAndSwap(prof,nil) {// Avoid a race condition.return }// Recompile the type. This is gonna be really slow, because// the compiler is slow, which is why we're doing it asynchronously.typeInfo.msgType.Store(typeInfo.msgType.Load().Recompile(typeInfo.prof.Load()))typeInfo.prof.Store(typeInfo.msgType.Load().NewProfile()) } }// Do something with msg.}
hyperpb is experimental software, and the API may change drastically beforev1. It currently implements all Protobuf language constructs. It does notimplement mutation of parsed messages, however.
hyperpb is currently only supported on 64-bit x86 and ARM targets (Go callstheseamd64 andarm64). The library will not build on other architectures,and PRs to add new architectures without a way to run tests for them in CI willbe rejected.
Support for 32-bit targets is unlikely to happen, even if we have test runners.hyperpb assumes a 64-bit general-purpose register width, and attempting totune for a 32-bit register size will require a 32-bit re-implementation of theparser VM.
Similarly, we assume little-endian in many places for performance, particularlybecause Protobuf's wire format is little-endian. Getting big-endian support willbe a lot of work and is unlikely to perform anywhere close to little-endian.
If you would like to try to use an architecture that we don't support, buildwith thehyperpb.unsupported tag. If it breaks, you get to keep both pieces:which is to say, issues stemming from use of this build tag will be closed.
For a detailed explanation of the implementation details ofhyperpb, seetheDESIGN.md file. Contributions that significantly change theparser will require benchmarks; you can run them withmake bench.
Offered under theApache 2 license.
About
10x faster dynamic Protobuf parsing in Go that’s even 3x faster than generated code.
Topics
Resources
License
Code of conduct
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Uh oh!
There was an error while loading.Please reload this page.
Contributors8
Uh oh!
There was an error while loading.Please reload this page.