Repository files navigation

A service command typically needs to update the database and publish messages/events.

– sending an e-mail message after placing an order

– sending an event about new client registration to the messaging system (RabbitMQ, Kafka, ...)

How to reliably/atomically update the database and publish messages/events?

We should implement theOutbox Pattern. A service that uses a relational database inserts messages/events into an outbox table in the same transaction.

(image fromhttps://microservices.io)

Outbox Pattern give usAt-Least-Once delivery. The receiver application may receive the message more than once.

Design a receiver to be anIdempotent Receiver

The term idempotent is used in mathematics to describe a function that produces the same result if it is applied to itself, i.e. f(x) = f(f(x)). In Messaging this concepts translates into a message that has the same effect whether it is received once or multiple times. This means that a message can safely be resent without causing any problems even if the receiver receives duplicates of the same message.

Example written in Golang, use Mysql as db and RabbitMQ as messaging system

Note: This is just a simple solution, the system has thousands of messages per sec should consider a tool likeDebezium

Components

1. App: Backend API
2. Relay: Read message from outbox table and publish to RabbitMQ
3. Worker: Doing backgound jobs

make build

1. Run the infras

make infras

2. Run the app, worker and relay. Open new terminal tab:

make dev

3. Cleaning

make clean

4. Send request. Open new terminal tab:

curl -X POST -H"Content-Type: application/json" -d'{"email":"test@example.com","name":"TESTTTTTT"}' http://localhost:3000/customers

You will see logs like:

outbox-demo-relay| 2021/08/04 09:37:16 Published messages: [fcb86b89-9404-48c7-9198-b85585ff8ab2]

outbox-demo-worker| 2021/08/04 09:37:16 Handling [CustomerCreated] - Payload:'{"id":"13db077f-b8b9-4512-9938-04f9aa00cae7","name":"TESTTTTTT","email":"test@example.com","CreatedAt":"2021-08-04T09:37:07.3053179Z","UpdatedAt":"0001-01-01T00:00:00Z"}'