Repository files navigation

Watermill is a Go library for working efficiently with message streams. It is intendedfor building event driven applications, enabling event sourcing, RPC over messages,sagas and basically whatever else comes to your mind. You can use conventional pub/subimplementations like Kafka or RabbitMQ, but also HTTP or PostgreSQL if that fits your use case.

• Easy to understand.
• Universal - event-driven architecture, messaging, stream processing, CQRS - use it for whatever you need.
• Fast (seeBenchmarks).
• Flexible with middlewares, plugins and Pub/Sub configurations.
• Resilient - using proven technologies and passing stress tests (seeStability).

Pick what you like the best or see in order:

1. Quickstart — learn by coding!
2. Follow theGetting Started guide.
3. See examples below.
4. Read the full documentation:https://watermill.io/

Go Event-Driven goes beyond Watermill Quickstart. You'll learn industry standard concepts and patterns like:

• Handling at-least-once delivery
• Asynchronous read models
• Events & Commands
• Observability
• Message ordering
• Sagas

• Basic
  • Your first app -start here!
  • Realtime feed
  • Router
  • Metrics
  • CQRS with protobuf
• Pub/Subs usage
  • These examples are part of theGetting started guide and show usage of a single Pub/Sub at a time.
• Real-world examples
  • Exactly-once delivery counter
  • Receiving webhooks
  • Sending webhooks
  • Synchronizing Databases
  • Persistent Event Log
  • Transactional Events
  • Real-time HTTP updates with Server-Sent Events
  • Real-time HTTP updates with Server-Sent Events and htmx
• Complete projects
  • NATS example with live code reloading
  • RabbitMQ, webhooks and Kafka integration

Building distributed and scalable services is rarely as easy as some may suggest. There is alot of hidden knowledge that comes with writing such systems. Just like you don't need to know thewhole TCP stack to create a HTTP REST server, you shouldn't need to study all of this knowledge tostart with building message-driven applications.

Watermill's goal is to make communication with messages as easy to use as HTTP routers. It providesthe tools needed to begin working with event-driven architecture and allows you to learn the detailson the go.

At the heart of Watermill there is one simple interface:

func(*Message) ([]*Message,error)

Your handler receives a message and decides whether to publish new message(s) or returnan error. What happens next is up to the middlewares you've chosen.

You can find more about our motivations in ourIntroducing Watermill blog post.

All publishers and subscribers have to implement an interface:

typePublisherinterface {Publish(topicstring,messages...*Message)errorClose()error}typeSubscriberinterface {Subscribe(ctx context.Context,topicstring) (<-chan*Message,error)Close()error}

Supported Pub/Subs:

• AMQP (RabbitMQ) Pub/Sub(github.com/ThreeDotsLabs/watermill-amqp/v3)
• AWS SNS/SQS Pub/Sub(github.com/ThreeDotsLabs/watermill-aws)
• Bolt Pub/Sub(github.com/ThreeDotsLabs/watermill-bolt)
• Firestore Pub/Sub(github.com/ThreeDotsLabs/watermill-firestore)
• Google Cloud Pub/Sub(github.com/ThreeDotsLabs/watermill-googlecloud/v2)
• HTTP Pub/Sub(github.com/ThreeDotsLabs/watermill-http/v2)
• io.Reader/io.Writer Pub/Sub(github.com/ThreeDotsLabs/watermill-io)
• Kafka Pub/Sub(github.com/ThreeDotsLabs/watermill-kafka/v3)
• NATS Jetstream Pub/Sub(github.com/ThreeDotsLabs/watermill-nats/v2)
• Redis Stream Pub/Sub(github.com/ThreeDotsLabs/watermill-redisstream)
• SQL (MySQL / PostgreSQL) Pub/Sub(github.com/ThreeDotsLabs/watermill-sql/v4)
• SQLite Pub/Sub (Beta)(github.com/ThreeDotsLabs/watermill-sqlite/)

All Pub/Subs implementation documentation can be found in thedocumentation.

Can't find your favorite Pub/Sub or library integration? CheckAwesome Watermill.

If you know another library or are an author of one, pleaseadd it to the list.

Please check ourcontributing guide.

Watermill v1.0.0 has been released and is production-ready. The public API is stable and will not change without changing the major version.

To ensure that all Pub/Subs are stable and safe to use in production, we created aset of tests that need to pass for each of the implementations before merging to master.All tests are also executed instress mode - that means that we are running all the tests20x in parallel.

All tests are run with the race condition detector enabled (-race flag in tests).

For more information about debugging tests, you should checktests troubleshooting guide.

Initial tools for benchmarking Pub/Subs can be found inwatermill-benchmark.

All benchmarks are being done on a single 16 CPU VM instance, running one binary and dependencies in Docker Compose.

These numbers are meant to serve as a rough estimate of how fast messages can be processed by different Pub/Subs.Keep in mind that the results can be vastly different, depending on the setup and configuration (both much lower and higher).

Here's the short version for message size of 16 bytes.

If you didn't find the answer to your question inthe documentation, feel free to ask us directly!

Please join us on the#watermill channel on theThree Dots Labs Discord.

It processes streams!

MIT License