protorange
package
This package is not in the latest version of its module.
Details
Validgo.mod file
The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
- Redistributable license
Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
- Tagged version
Modules with tagged versions give importers more predictable builds.
- Stable version
When a project reaches major version v1 it is considered stable.
- Learn more about best practices
Repository
Links
Open Source Insights
Documentation¶
Overview¶
Package protorange provides functionality to traverse a message value.
Example (DiscardUnknown)¶
Range through every message and clear the unknown fields.
package mainimport ("fmt""google.golang.org/protobuf/reflect/protopath""google.golang.org/protobuf/reflect/protorange""google.golang.org/protobuf/reflect/protoreflect""google.golang.org/protobuf/testing/protopack"newspb "google.golang.org/protobuf/internal/testprotos/news")func main() {// Populate the article with unknown fields.m := &newspb.Article{}m.ProtoReflect().SetUnknown(protopack.Message{protopack.Tag{1000, protopack.BytesType}, protopack.String("Hello, world!"),}.Marshal())fmt.Println("has unknown fields?", len(m.ProtoReflect().GetUnknown()) > 0)// Range through the message and clear all unknown fields.fmt.Println("clear unknown fields")protorange.Range(m.ProtoReflect(), func(proto protopath.Values) error {m, ok := proto.Index(-1).Value.Interface().(protoreflect.Message)if ok && len(m.GetUnknown()) > 0 {m.SetUnknown(nil)}return nil})fmt.Println("has unknown fields?", len(m.ProtoReflect().GetUnknown()) > 0)}Output:has unknown fields? trueclear unknown fieldshas unknown fields? false
Example (FormatText)¶
Implement a basic text formatter by ranging through all populated valuesin a message in depth-first order.
package mainimport ("fmt""time""google.golang.org/protobuf/proto""google.golang.org/protobuf/reflect/protopath""google.golang.org/protobuf/reflect/protorange""google.golang.org/protobuf/reflect/protoreflect""google.golang.org/protobuf/types/known/anypb""google.golang.org/protobuf/types/known/timestamppb"newspb "google.golang.org/protobuf/internal/testprotos/news")func mustMarshal(m proto.Message) []byte {b, err := proto.Marshal(m)if err != nil {panic(err)}return b}func main() {m := &newspb.Article{Author: "Brad Fitzpatrick",Date: timestamppb.New(time.Date(2018, time.February, 16, 0, 0, 0, 0, time.UTC)),Title: "Go 1.10 is released",Content: "Happy Friday, happy weekend! Today the Go team is happy to announce the release of Go 1.10...",Status: newspb.Article_PUBLISHED,Tags: []string{"go1.10", "release"},Attachments: []*anypb.Any{{TypeUrl: "google.golang.org.KeyValueAttachment",Value: mustMarshal(&newspb.KeyValueAttachment{Name: "checksums.txt",Data: map[string]string{"go1.10.src.tar.gz": "07cbb9d0091b846c6aea40bf5bc0cea7","go1.10.darwin-amd64.pkg": "cbb38bb6ff6ea86279e01745984445bf","go1.10.linux-amd64.tar.gz": "6b3d0e4a5c77352cf4275573817f7566","go1.10.windows-amd64.msi": "57bda02030f58f5d2bf71943e1390123",},}),}},}// Print a message in a humanly readable format.var indent []byteprotorange.Options{Stable: true,}.Range(m.ProtoReflect(),func(p protopath.Values) error {// Print the key.var fd protoreflect.FieldDescriptorlast := p.Index(-1)beforeLast := p.Index(-2)switch last.Step.Kind() {case protopath.FieldAccessStep:fd = last.Step.FieldDescriptor()fmt.Printf("%s%s: ", indent, fd.Name())case protopath.ListIndexStep:fd = beforeLast.Step.FieldDescriptor() // lists always appear in the context of a repeated fieldfmt.Printf("%s%d: ", indent, last.Step.ListIndex())case protopath.MapIndexStep:fd = beforeLast.Step.FieldDescriptor() // maps always appear in the context of a repeated fieldfmt.Printf("%s%v: ", indent, last.Step.MapIndex().Interface())case protopath.AnyExpandStep:fmt.Printf("%s[%v]: ", indent, last.Value.Message().Descriptor().FullName())case protopath.UnknownAccessStep:fmt.Printf("%s?: ", indent)}// Starting printing the value.switch v := last.Value.Interface().(type) {case protoreflect.Message:fmt.Printf("{\n")indent = append(indent, '\t')case protoreflect.List:fmt.Printf("[\n")indent = append(indent, '\t')case protoreflect.Map:fmt.Printf("{\n")indent = append(indent, '\t')case protoreflect.EnumNumber:var ev protoreflect.EnumValueDescriptorif fd != nil {ev = fd.Enum().Values().ByNumber(v)}if ev != nil {fmt.Printf("%v\n", ev.Name())} else {fmt.Printf("%v\n", v)}case string, []byte:fmt.Printf("%q\n", v)default:fmt.Printf("%v\n", v)}return nil},func(p protopath.Values) error {// Finish printing the value.last := p.Index(-1)switch last.Value.Interface().(type) {case protoreflect.Message:indent = indent[:len(indent)-1]fmt.Printf("%s}\n", indent)case protoreflect.List:indent = indent[:len(indent)-1]fmt.Printf("%s]\n", indent)case protoreflect.Map:indent = indent[:len(indent)-1]fmt.Printf("%s}\n", indent)}return nil},)}Output:{author: "Brad Fitzpatrick"date: {seconds: 1518739200}title: "Go 1.10 is released"content: "Happy Friday, happy weekend! Today the Go team is happy to announce the release of Go 1.10..."attachments: [0: {[google.golang.org.KeyValueAttachment]: {name: "checksums.txt"data: {go1.10.darwin-amd64.pkg: "cbb38bb6ff6ea86279e01745984445bf"go1.10.linux-amd64.tar.gz: "6b3d0e4a5c77352cf4275573817f7566"go1.10.src.tar.gz: "07cbb9d0091b846c6aea40bf5bc0cea7"go1.10.windows-amd64.msi: "57bda02030f58f5d2bf71943e1390123"}}}]tags: [0: "go1.10"1: "release"]status: PUBLISHED}
Example (PrintPaths)¶
Print the relative paths as Range iterates through a messagein a depth-first order.
package mainimport ("fmt""time""google.golang.org/protobuf/proto""google.golang.org/protobuf/reflect/protopath""google.golang.org/protobuf/reflect/protorange""google.golang.org/protobuf/types/known/anypb""google.golang.org/protobuf/types/known/timestamppb"newspb "google.golang.org/protobuf/internal/testprotos/news")func mustMarshal(m proto.Message) []byte {b, err := proto.Marshal(m)if err != nil {panic(err)}return b}func main() {m := &newspb.Article{Author: "Russ Cox",Date: timestamppb.New(time.Date(2019, time.November, 8, 0, 0, 0, 0, time.UTC)),Title: "Go Turns 10",Content: "Happy birthday, Go! This weekend we celebrate the 10th anniversary of the Go release...",Status: newspb.Article_PUBLISHED,Tags: []string{"community", "birthday"},Attachments: []*anypb.Any{{TypeUrl: "google.golang.org.BinaryAttachment",Value: mustMarshal(&newspb.BinaryAttachment{Name: "gopher-birthday.png",Data: []byte("<binary data>"),}),}},}// Traverse over all reachable values and print the path.protorange.Range(m.ProtoReflect(), func(p protopath.Values) error {fmt.Println(p.Path[1:])return nil})}Output:.author.date.date.seconds.title.content.status.tags.tags[0].tags[1].attachments.attachments[0].attachments[0].(google.golang.org.BinaryAttachment).attachments[0].(google.golang.org.BinaryAttachment).name.attachments[0].(google.golang.org.BinaryAttachment).data
Example (SanitizeStrings)¶
Scan all protobuf string values for a sensitive word and replace it witha suitable alternative.
package mainimport ("fmt""strings""time""google.golang.org/protobuf/encoding/protojson""google.golang.org/protobuf/proto""google.golang.org/protobuf/reflect/protopath""google.golang.org/protobuf/reflect/protorange""google.golang.org/protobuf/reflect/protoreflect""google.golang.org/protobuf/types/known/anypb""google.golang.org/protobuf/types/known/timestamppb"newspb "google.golang.org/protobuf/internal/testprotos/news")func mustMarshal(m proto.Message) []byte {b, err := proto.Marshal(m)if err != nil {panic(err)}return b}func main() {m := &newspb.Article{Author: "Hermione Granger",Date: timestamppb.New(time.Date(1998, time.May, 2, 0, 0, 0, 0, time.UTC)),Title: "Harry Potter vanquishes Voldemort once and for all!",Content: "In a final duel between Harry Potter and Lord Voldemort earlier this evening...",Tags: []string{"HarryPotter", "LordVoldemort"},Attachments: []*anypb.Any{{TypeUrl: "google.golang.org.KeyValueAttachment",Value: mustMarshal(&newspb.KeyValueAttachment{Name: "aliases.txt",Data: map[string]string{"Harry Potter": "The Boy Who Lived","Tom Riddle": "Lord Voldemort",},}),}},}protorange.Range(m.ProtoReflect(), func(p protopath.Values) error {const (sensitive = "Voldemort"alternative = "[He-Who-Must-Not-Be-Named]")// Check if there is a sensitive word to redact.last := p.Index(-1)s, ok := last.Value.Interface().(string)if !ok || !strings.Contains(s, sensitive) {return nil}s = strings.Replace(s, sensitive, alternative, -1)// Store the redacted string back into the message.beforeLast := p.Index(-2)switch last.Step.Kind() {case protopath.FieldAccessStep:m := beforeLast.Value.Message()fd := last.Step.FieldDescriptor()m.Set(fd, protoreflect.ValueOfString(s))case protopath.ListIndexStep:ls := beforeLast.Value.List()i := last.Step.ListIndex()ls.Set(i, protoreflect.ValueOfString(s))case protopath.MapIndexStep:ms := beforeLast.Value.Map()k := last.Step.MapIndex()ms.Set(k, protoreflect.ValueOfString(s))}return nil})fmt.Println(protojson.Format(m))}Output:{ "author": "Hermione Granger", "date": "1998-05-02T00:00:00Z", "title": "Harry Potter vanquishes [He-Who-Must-Not-Be-Named] once and for all!", "content": "In a final duel between Harry Potter and Lord [He-Who-Must-Not-Be-Named] earlier this evening...", "tags": [ "HarryPotter", "Lord[He-Who-Must-Not-Be-Named]" ], "attachments": [ { "@type": "google.golang.org.KeyValueAttachment", "name": "aliases.txt", "data": { "Harry Potter": "The Boy Who Lived", "Tom Riddle": "Lord [He-Who-Must-Not-Be-Named]" } } ]}
Index¶
Examples¶
Constants¶
This section is empty.
Variables¶
var (// Break breaks traversal of children in the current value.// It has no effect when traversing values that are not composite types// (e.g., messages, lists, and maps).Break =errors.New("break traversal of children in current value")// Terminate terminates the entire range operation.// All necessary Pop operations continue to be called.Terminate =errors.New("terminate range operation"))
Functions¶
funcRange¶
Range performs a depth-first traversal over reachable values in a message.
SeeOptions.Range for details.
Types¶
typeOptions¶
type Options struct {// Stable specifies whether to visit message fields and map entries// in a stable ordering. If false, then the ordering is undefined and// may be non-deterministic.//// Message fields are visited in ascending order by field number.// Map entries are visited in ascending order, where// boolean keys are ordered such that false sorts before true,// numeric keys are ordered based on the numeric value, and// string keys are lexicographically ordered by Unicode codepoints.Stablebool// Resolver is used for looking up types when expanding google.protobuf.Any// messages. If nil, this defaults to using protoregistry.GlobalTypes.// To prevent expansion of Any messages, pass an empty protoregistry.Types:////Options{Resolver: (*protoregistry.Types)(nil)}//Resolver interface {protoregistry.ExtensionTypeResolverprotoregistry.MessageTypeResolver}}Options configures traversal of a message value tree.
func (Options)Range¶
Range performs a depth-first traversal over reachable values in a message.The first push and the last pop are to push/pop aprotopath.Root step.If push or pop return any non-nil error (other thanBreak orTerminate),it terminates the traversal and is returned by Range.
The rules for traversing a message is as follows:
For messages, iterate over every populated known and extension field.Each field is preceded by a push of aprotopath.FieldAccess step,followed by recursive application of the rules on the field value,and succeeded by a pop of that step.If the message has unknown fields, then push anprotopath.UnknownAccess stepfollowed immediately by pop of that step.
As an exception to the above rule, if the current message is agoogle.protobuf.Any message, expand the underlying message (if resolvable).The expanded message is preceded by a push of aprotopath.AnyExpand step,followed by recursive application of the rules on the underlying message,and succeeded by a pop of that step. Mutations to the expanded messageare written back to the Any message when popping back out.
For lists, iterate over every element. Each element is preceded by a pushof aprotopath.ListIndex step, followed by recursive application of the ruleson the list element, and succeeded by a pop of that step.
For maps, iterate over every entry. Each entry is preceded by a pushof aprotopath.MapIndex step, followed by recursive application of the ruleson the map entry value, and succeeded by a pop of that step.
Mutations should only be made to the last value, otherwise the effects ontraversal will be undefined. If the mutation is made to the last valueduring to a push, then the effects of the mutation will affect traversal.For example, if the last value is currently a message, and the push functionpopulates a few fields in that message, then the newly modified fieldswill be traversed.
Theprotopath.Values provided to push functions is only valid until thecorresponding pop call and the values provided to a pop call is only validfor the duration of the pop call itself.
Source Files