How to use Protobuf with Go?

Have you ever been in a situation where you do not know the structure of data that services are either consuming or publishing?

Did you know that you have to write a schema only once which defines the structure data and the same schema can be used as validation before sending & receiving data usingProtocol Buffers?

In this tutorial, you will learn about using Protobuf. You will send & receive a message asynchronously using Protobuf withRabbitMQ.

Table Of Contents

• Prerequisites
• Protobuf
• Bootstrapping Project
• Creating Protobuf Messages
• Publishing a Message
• Consuming a Message
• Conclusion

Prerequisites

This tutorial will be a hands-on demonstration. If you’d like to follow along, be sure you have the following:

• You have runningDocker Composer in your machine.
• You have a basic understanding ofDocker,GO,Rabbitmq.

Protobuf

Protobufs are the Messaging Contracts. They are schema that describes the structure of the message. These files are written in.proto extension which is language-neutral i.e write once & use anywhere. Proto Messages are the structured schema that is inside.protofiles

You will write Proto Messages and with the compiler calledProto Compiler, you can generate the code which will have a way to serialize & deserialize data into that format.

There are many benefits of using Messaging Contracts like Protobuf :

1. With these Proto Messages, you will know what type of data you are sending & receiving
2. Serves as a documentation
3. Supports validation for serializing & deserializing
4. Proto messages are language-neutral and it supports various programming languages which means you can publish messages from Go & start consuming them from Php / Node etc

Bootstrapping Project

For creating a new test project follow the steps below:

• CopyDockerfile &docker-compose.yaml fromthis repository which will dockerized Go source code, Proto compiler & RabbitMQ
• Create a filepublish.go where you will write some code to publish events
• Create a fileconsume.go where you will write some code to consume events

Creating Protobuf Messages

In an application, while you sending messages you will serialize data using the definition from Proto Messages. And when consuming messages you will deserialize them using the same Proto Messages. Let's create a Proto Message so your services can use them to send and receive messages.

1. Create a foldermessages and inside that folder create a file calleduser_registered.go
2. Insideuser_registered.go copy the following codes. Let's understand the code

• syntax defines which proto version you are going to use
• go_package defines the path where all your generated Protobuf code will be into
• `import “validate/validate.proto”; it is importing the validator so that you can validate message using various rules without writing it by yourself
• All message starts with a message keyword followed by name of the message. Inside the block, you can write key-value pairs where the key defines name & type & the value defines its order & rules if there are any. In our case, you are defining the first fielduserId which is an integer and should be greater than 0 and the next is an email which is a string and should not be empty.

\3. Runmake build-proto on your project root. This command will generate two filesuser_registered.pb.go and user_registered.pb.validate.go that define type & includes all the validation logic that you specified on the text which you can later use on your code while serializing & deserializing.

Publishing a Message

As you have successfully created a Proto Message. Let's use the same auto-generated code while sending messages to other services for this you need to write a way to publish messages. Follow the following steps:

1. Create a directorypkg and a filerabbitmq.go inside pkg dir where you will make a connection with RabbitMQ. The following code the functiongetConnection is importing library and making a connection with RabbitMQ server and upon error functionfailOnError is terminating the application with an error.

\2. After successfully making a connection, write a function that will publish a message to RabbitMQ. The following code acceptsqueueName and payload as amessage which will write a message to default exchange to JSON format.

\3. Createpublish.go which will import publish function fromrabbitmq.go to send data. The following code is using a struct that was generated by the proto using the proto compiler and it is encoding them into JSON after that it is calling to publish the function that you created earlier to send data toUserRegisteredqueue.

Try runningmake publish to verify that your code is actually working if you seeData has been published the output that means you were successful in publishing a message.

Consuming a Message

Until now you defined a photo message then you created arabbitmq.goto create a way to publish messages & you used the same functionpublish.go to publish messages to RabbitMq. Now let's consume the message that you published earlier.

1. Insiderabbit. go add a function that will help in consuming a message. The following code makes a connection to RabbitMq & returns a channel that you can leverage to listen to incoming messages.

\2. Insideconsume.go listen to all the messages and then print them. Let's understand the following code :

• The main function is listening to message by creating a consuming client. While deserializing message it is importingUserRegistered struct from the auto-generated code to define the type of data. There might be cases where the producer might send invalid data due to various reasons so it is better to validate that data you are consuming are as per the contracts that you specified.
• validateMessage accepts the JSON data and tries to validate as per theuser_registered.proto definition and if the data breaks any validation rules it will throw a validation error

Conclusion

Throughout this tutorial, you have learned Protobuf & how to use them with GO & RabbitMQ.

Protobuf is one of the ways to implement Messaging Contracts there are more options likePact,AsyncApi

Now with this knowledge, you can use Protobuf on any programming language to send & receive the data. You have also learned that it was easier to predict the data that you are consuming or publishing.

Let me know if you have any comments /suggestions/feedback.

Source code:https://github.com/ujwaldhakal/golang-protobuf
Ref:https://stackshare.io/protobuf