Overview¶

Package gob manages streams of gobs - binary values exchanged between anEncoder (transmitter) and aDecoder (receiver). A typical use is transportingarguments and results of remote procedure calls (RPCs) such as those provided bynet/rpc.

The implementation compiles a custom codec for each data type in the stream andis most efficient when a singleEncoder is used to transmit a stream of values,amortizing the cost of compilation.

Basics¶

A stream of gobs is self-describing. Each data item in the stream is preceded bya specification of its type, expressed in terms of a small set of predefinedtypes. Pointers are not transmitted, but the things they point to aretransmitted; that is, the values are flattened. Nil pointers are not permitted,as they have no value. Recursive types work fine, butrecursive values (data with cycles) are problematic. This may change.

To use gobs, create anEncoder and present it with a series of data items asvalues or addresses that can be dereferenced to values. TheEncoder makes sureall type information is sent before it is needed. At the receive side, aDecoder retrieves values from the encoded stream and unpacks them into localvariables.

Types and Values¶

The source and destination values/types need not correspond exactly. For structs,fields (identified by name) that are in the source but absent from the receivingvariable will be ignored. Fields that are in the receiving variable but missingfrom the transmitted type or value will be ignored in the destination. If a fieldwith the same name is present in both, their types must be compatible. Both thereceiver and transmitter will do all necessary indirection and dereferencing toconvert between gobs and actual Go values. For instance, a gob type that isschematically,

struct { A, B int }

can be sent from or received into any of these Go types:

struct { A, B int }// the same*struct { A, B int }// extra indirection of the structstruct { *A, **B int }// extra indirection of the fieldsstruct { A, B int64 }// different concrete value type; see below

It may also be received into any of these:

struct { A, B int }// the samestruct { B, A int }// ordering doesn't matter; matching is by namestruct { A, B, C int }// extra field (C) ignoredstruct { B int }// missing field (A) ignored; data will be droppedstruct { B, C int }// missing field (A) ignored; extra field (C) ignored.

Attempting to receive into these types will draw a decode error:

struct { A int; B uint }// change of signedness for Bstruct { A int; B float }// change of type for Bstruct { }// no field names in commonstruct { C, D int }// no field names in common

Integers are transmitted two ways: arbitrary precision signed integers orarbitrary precision unsigned integers. There is no int8, int16 etc.discrimination in the gob format; there are only signed and unsigned integers. Asdescribed below, the transmitter sends the value in a variable-length encoding;the receiver accepts the value and stores it in the destination variable.Floating-point numbers are always sent using IEEE 754 64-bit precision (seebelow).

Signed integers may be received into any signed integer variable: int, int16, etc.;unsigned integers may be received into any unsigned integer variable; and floatingpoint values may be received into any floating point variable. However,the destination variable must be able to represent the value or the decodeoperation will fail.

Structs, arrays and slices are also supported. Structs encode and decode onlyexported fields. Strings and arrays of bytes are supported with a special,efficient representation (see below). When a slice is decoded, if the existingslice has capacity the slice will be extended in place; if not, a new array isallocated. Regardless, the length of the resulting slice reports the number ofelements decoded.

In general, if allocation is required, the decoder will allocate memory. If not,it will update the destination variables with values read from the stream. It doesnot initialize them first, so if the destination is a compound value such as amap, struct, or slice, the decoded values will be merged elementwise into theexisting variables.

Functions and channels will not be sent in a gob. Attempting to encode such a valueat the top level will fail. A struct field of chan or func type is treated exactlylike an unexported field and is ignored.

Gob can encode a value of any type implementing theGobEncoder orencoding.BinaryMarshaler interfaces by calling the corresponding method,in that order of preference.

Gob can decode a value of any type implementing theGobDecoder orencoding.BinaryUnmarshaler interfaces by calling the corresponding method,again in that order of preference.

Encoding Details¶

This section documents the encoding, details that are not important for mostusers. Details are presented bottom-up.

An unsigned integer is sent one of two ways. If it is less than 128, it is sentas a byte with that value. Otherwise it is sent as a minimal-length big-endian(high byte first) byte stream holding the value, preceded by one byte holding thebyte count, negated. Thus 0 is transmitted as (00), 7 is transmitted as (07) and256 is transmitted as (FE 01 00).

A boolean is encoded within an unsigned integer: 0 for false, 1 for true.

A signed integer, i, is encoded within an unsigned integer, u. Within u, bits 1upward contain the value; bit 0 says whether they should be complemented uponreceipt. The encode algorithm looks like this:

var u uintif i < 0 {u = (^uint(i) << 1) | 1 // complement i, bit 0 is 1} else {u = (uint(i) << 1) // do not complement i, bit 0 is 0}encodeUnsigned(u)

The low bit is therefore analogous to a sign bit, but making it the complement bitinstead guarantees that the largest negative integer is not a special case. Forexample, -129=^128=(^256>>1) encodes as (FE 01 01).

Floating-point numbers are always sent as a representation of a float64 value.That value is converted to a uint64 usingmath.Float64bits. The uint64 is thenbyte-reversed and sent as a regular unsigned integer. The byte-reversal means theexponent and high-precision part of the mantissa go first. Since the low bits areoften zero, this can save encoding bytes. For instance, 17.0 is encoded in onlythree bytes (FE 31 40).

Strings and slices of bytes are sent as an unsigned count followed by that manyuninterpreted bytes of the value.

All other slices and arrays are sent as an unsigned count followed by that manyelements using the standard gob encoding for their type, recursively.

Maps are sent as an unsigned count followed by that many key, elementpairs. Empty but non-nil maps are sent, so if the receiver has not allocatedone already, one will always be allocated on receipt unless the transmitted mapis nil and not at the top level.

In slices and arrays, as well as maps, all elements, even zero-valued elements,are transmitted, even if all the elements are zero.

Structs are sent as a sequence of (field number, field value) pairs. The fieldvalue is sent using the standard gob encoding for its type, recursively. If afield has the zero value for its type (except for arrays; see above), it is omittedfrom the transmission. The field number is defined by the type of the encodedstruct: the first field of the encoded type is field 0, the second is field 1,etc. When encoding a value, the field numbers are delta encoded for efficiencyand the fields are always sent in order of increasing field number; the deltas aretherefore unsigned. The initialization for the delta encoding sets the fieldnumber to -1, so an unsigned integer field 0 with value 7 is transmitted as unsigneddelta = 1, unsigned value = 7 or (01 07). Finally, after all the fields have beensent a terminating mark denotes the end of the struct. That mark is a delta=0value, which has representation (00).

Interface types are not checked for compatibility; all interface types aretreated, for transmission, as members of a single "interface" type, analogous toint or []byte - in effect they're all treated as interface{}. Interface valuesare transmitted as a string identifying the concrete type being sent (a namethat must be pre-defined by callingRegister), followed by a byte count of thelength of the following data (so the value can be skipped if it cannot bestored), followed by the usual encoding of concrete (dynamic) value stored inthe interface value. (A nil interface value is identified by the empty stringand transmits no value.) Upon receipt, the decoder verifies that the unpackedconcrete item satisfies the interface of the receiving variable.

If a value is passed toEncoder.Encode and the type is not a struct (or pointer to struct,etc.), for simplicity of processing it is represented as a struct of one field.The only visible effect of this is to encode a zero byte after the value, just asafter the last field of an encoded struct, so that the decode algorithm knows whenthe top-level value is complete.

The representation of types is described below. When a type is defined on a givenconnection between anEncoder andDecoder, it is assigned a signed integer typeid. WhenEncoder.Encode(v) is called, it makes sure there is an id assigned forthe type of v and all its elements and then it sends the pair (typeid, encoded-v)where typeid is the type id of the encoded type of v and encoded-v is the gobencoding of the value v.

To define a type, the encoder chooses an unused, positive type id and sends thepair (-type id, encoded-type) where encoded-type is the gob encoding of a wireTypedescription, constructed from these types:

type wireType struct {ArrayT           *arrayTypeSliceT           *sliceTypeStructT          *structTypeMapT             *mapTypeGobEncoderT      *gobEncoderTypeBinaryMarshalerT *gobEncoderTypeTextMarshalerT   *gobEncoderType}type arrayType struct {CommonTypeElem typeIdLen  int}type CommonType struct {Name string // the name of the struct typeId  int    // the id of the type, repeated so it's inside the type}type sliceType struct {CommonTypeElem typeId}type structType struct {CommonTypeField []fieldType // the fields of the struct.}type fieldType struct {Name string // the name of the field.Id   int    // the type id of the field, which must be already defined}type mapType struct {CommonTypeKey  typeIdElem typeId}type gobEncoderType struct {CommonType}

If there are nested type ids, the types for all inner type ids must be definedbefore the top-level type id is used to describe an encoded-v.

For simplicity in setup, the connection is defined to understand these types apriori, as well as the basic gob types int, uint, etc. Their ids are:

bool        1int         2uint        3float       4[]byte      5string      6complex     7interface   8// gap for reserved ids.WireType    16ArrayType   17CommonType  18SliceType   19StructType  20FieldType   21// 22 is slice of fieldType.MapType     23

Finally, each message created by a call to Encode is preceded by an encodedunsigned integer count of the number of bytes remaining in the message. Afterthe initial type name, interface values are wrapped the same way; in effect, theinterface value acts like a recursive invocation of Encode.

In summary, a gob stream looks like

(byteCount (-type id, encoding of a wireType)* (type id, encoding of a value))*

where * signifies zero or more repetitions and the type id of a value mustbe predefined or be defined before the value in the stream.

Compatibility: Any future changes to the package will endeavor to maintaincompatibility with streams encoded using previous versions. That is, any releasedversion of this package should be able to decode data written with any previouslyreleased version, subject to issues such as security fixes. See the Go compatibilitydocument for background:https://golang.org/doc/go1compat

See "Gobs of data" for a design discussion of the gob wire format:https://blog.golang.org/gobs-of-data

Security¶

This package is not designed to be hardened against adversarial inputs, and isoutside the scope ofhttps://go.dev/security/policy. In particular, theDecoderdoes only basic sanity checking on decoded input sizes, and its limits are notconfigurable. Care should be taken when decoding gob data from untrustedsources, which may consume significant resources.

package mainimport ("bytes""encoding/gob""fmt""log")type P struct {X, Y, Z intName    string}type Q struct {X, Y *int32Name string}// This example shows the basic usage of the package: Create an encoder,// transmit some values, receive them with a decoder.func main() {// Initialize the encoder and decoder. Normally enc and dec would be// bound to network connections and the encoder and decoder would// run in different processes.var network bytes.Buffer        // Stand-in for a network connectionenc := gob.NewEncoder(&network) // Will write to network.dec := gob.NewDecoder(&network) // Will read from network.// Encode (send) some values.err := enc.Encode(P{3, 4, 5, "Pythagoras"})if err != nil {log.Fatal("encode error:", err)}err = enc.Encode(P{1782, 1841, 1922, "Treehouse"})if err != nil {log.Fatal("encode error:", err)}// Decode (receive) and print the values.var q Qerr = dec.Decode(&q)if err != nil {log.Fatal("decode error 1:", err)}fmt.Printf("%q: {%d, %d}\n", q.Name, *q.X, *q.Y)err = dec.Decode(&q)if err != nil {log.Fatal("decode error 2:", err)}fmt.Printf("%q: {%d, %d}\n", q.Name, *q.X, *q.Y)}

Output:"Pythagoras": {3, 4}"Treehouse": {1782, 1841}

package mainimport ("bytes""encoding/gob""fmt""log")// The Vector type has unexported fields, which the package cannot access.// We therefore write a BinaryMarshal/BinaryUnmarshal method pair to allow us// to send and receive the type with the gob package. These interfaces are// defined in the "encoding" package.// We could equivalently use the locally defined GobEncode/GobDecoder// interfaces.type Vector struct {x, y, z int}func (v Vector) MarshalBinary() ([]byte, error) {// A simple encoding: plain text.var b bytes.Bufferfmt.Fprintln(&b, v.x, v.y, v.z)return b.Bytes(), nil}// UnmarshalBinary modifies the receiver so it must take a pointer receiver.func (v *Vector) UnmarshalBinary(data []byte) error {// A simple encoding: plain text.b := bytes.NewBuffer(data)_, err := fmt.Fscanln(b, &v.x, &v.y, &v.z)return err}// This example transmits a value that implements the custom encoding and decoding methods.func main() {var network bytes.Buffer // Stand-in for the network.// Create an encoder and send a value.enc := gob.NewEncoder(&network)err := enc.Encode(Vector{3, 4, 5})if err != nil {log.Fatal("encode:", err)}// Create a decoder and receive a value.dec := gob.NewDecoder(&network)var v Vectorerr = dec.Decode(&v)if err != nil {log.Fatal("decode:", err)}fmt.Println(v)}

Output:{3 4 5}

package mainimport ("bytes""encoding/gob""fmt""log""math")type Point struct {X, Y int}func (p Point) Hypotenuse() float64 {return math.Hypot(float64(p.X), float64(p.Y))}type Pythagoras interface {Hypotenuse() float64}// This example shows how to encode an interface value. The key// distinction from regular types is to register the concrete type that// implements the interface.func main() {var network bytes.Buffer // Stand-in for the network.// We must register the concrete type for the encoder and decoder (which would// normally be on a separate machine from the encoder). On each end, this tells the// engine which concrete type is being sent that implements the interface.gob.Register(Point{})// Create an encoder and send some values.enc := gob.NewEncoder(&network)for i := 1; i <= 3; i++ {interfaceEncode(enc, Point{3 * i, 4 * i})}// Create a decoder and receive some values.dec := gob.NewDecoder(&network)for i := 1; i <= 3; i++ {result := interfaceDecode(dec)fmt.Println(result.Hypotenuse())}}// interfaceEncode encodes the interface value into the encoder.func interfaceEncode(enc *gob.Encoder, p Pythagoras) {// The encode will fail unless the concrete type has been// registered. We registered it in the calling function.// Pass pointer to interface so Encode sees (and hence sends) a value of// interface type. If we passed p directly it would see the concrete type instead.// See the blog post, "The Laws of Reflection" for background.err := enc.Encode(&p)if err != nil {log.Fatal("encode:", err)}}// interfaceDecode decodes the next interface value from the stream and returns it.func interfaceDecode(dec *gob.Decoder) Pythagoras {// The decode will fail unless the concrete type on the wire has been// registered. We registered it in the calling function.var p Pythagoraserr := dec.Decode(&p)if err != nil {log.Fatal("decode:", err)}return p}

Output:51015

Index¶

• func Register(value any)
• func RegisterName(name string, value any)
• type CommonType
• type Decoder
• • func NewDecoder(r io.Reader) *Decoder
• • func (dec *Decoder) Decode(e any) error
  • func (dec *Decoder) DecodeValue(v reflect.Value) error
• type Encoder
• • func NewEncoder(w io.Writer) *Encoder
• • func (enc *Encoder) Encode(e any) error
  • func (enc *Encoder) EncodeValue(value reflect.Value) error
• type GobDecoder
• type GobEncoder

Examples¶

• Package (Basic)
• Package (EncodeDecode)
• Package (Interface)

This section is empty.

This section is empty.