Repository files navigation

Event store implemented in Elixir. UsesPostgreSQL as the underlying storage engine.

Requires Elixir v1.10 and PostgreSQL v9.5 or newer.

EventStore supportsrunning on a cluster of nodes.

• Changelog
• Wiki
• Frequently asked questions
• Getting help
• Latest published Hex package &documentation

MIT License

This README and the following guides follow themaster branch which may not be the currently published version.Read docs for the latest published version of EventStore on Hex.

• Getting started
  • Using an existing database
  • Reset an existing database
  • Initialize a database using an Elixir release
  • Using Postgres schemas
  • Event data and metadata data type
    • Using thejsonb data type
  • Using with PgBouncer
• Using the EventStore
  • Writing to a stream
    • Appending events to an existing stream
  • Reading from a stream
  • Reading from all streams
  • Stream from all streams
  • Linking events between streams
  • Deleting streams
• Subscriptions
  • Transient subscriptions
  • Persistent subscriptions
    • Acknowledge received events
    • Subscription concurrency
    • Example persistent subscriber
    • Deleting a persistent subscription
• Running on a cluster
• Event serialization
• Upgrading an EventStore
• Used in production?
• Backup and administration
• Benchmarking performance
• Contributing
  • Contributors
• Need help?

Define an event store module:

defmoduleMyEventStoredouseEventStore,otp_app::my_append

Start the event store:

{:ok,_pid}=MyEventStore.start_link()

Create one or more event structs to be persisted (serialized to JSON by default):

defmoduleExampleEventdodefstruct[:key]end

Append events to a stream:

stream_uuid=EventStore.UUID.uuid4()expected_version=0events=[%EventStore.EventData{event_type:"Elixir.ExampleEvent",data:%ExampleEvent{key:"value"},metadata:%{user:"someuser@example.com"}}]:ok=MyEventStore.append_to_stream(stream_uuid,expected_version,events)

Read all events from a single stream, starting at the stream's first event:

{:ok,events}=MyEventStore.read_stream_forward(stream_uuid)

More:Usage guide

Subscribe to events appended to all streams:

{:ok,subscription}=MyEventStore.subscribe_to_all_streams("example_subscription",self())# Wait for the subscription confirmationreceivedo{:subscribed,^subscription}->IO.puts("Successfully subscribed to all streams")end# Receive a batch of events appended to the event storereceivedo{:events,events}->IO.puts("Received events:#{inspectevents}")# Acknowledge successful receipt of events:ok=MyEventStore.ack(subscription,events)end

In production use you would use aGenServer subscriber process and thehandle_info/2 callback to receive events.

More:Subscriptions guide

Yes, this event store is being used in production.

PostgreSQL is used for the underlying storage. Providing guarantees to store data securely. It is ACID-compliant and transactional. PostgreSQL has a proven architecture. A strong reputation for reliability, data integrity, and correctness.

You can use any standard PostgreSQL tool to manage the event store data:

• Backup and restore.
• Continuous archiving and Point-in-Time Recovery (PITR).

Run the benchmark suite using mix with thebench environment, as configured inconfig/bench.exs. Logging is disabled for benchmarking.

MIX_ENV=bench mix do es.reset, app.start, bench

Example output:

## AppendEventsBenchbenchmark name                         iterations   average timeappend events, single writer                  100   20288.68 µs/opappend events, 10 concurrent writers           10   127416.90 µs/opappend events, 20 concurrent writers            5   376836.60 µs/opappend events, 50 concurrent writers            2   582350.50 µs/op## ReadEventsBenchbenchmark name                         iterations   average timeread events, single reader                    500   3674.93 µs/opread events, 10 concurrent readers             50   44653.98 µs/opread events, 20 concurrent readers             20   73927.55 µs/opread events, 50 concurrent readers             10   188244.80 µs/op## SubscribeToStreamBenchbenchmark name                         iterations   average timesubscribe to stream, 1 subscription           100   27687.97 µs/opsubscribe to stream, 10 subscriptions          50   56047.72 µs/opsubscribe to stream, 20 subscriptions          10   194164.40 µs/opsubscribe to stream, 50 subscriptions           5   320435.40 µs/op

After running two benchmarks you can compare the runs:

MIX_ENV=bench mix bench.cmp -d percent

You can also produce an HTML page containing a graph comparing benchmark runs:

MIX_ENV=bench mix bench.graph

Taking the above example output, the append events benchmark is for writing 100 events in a single batch. That's what the µs/op average time is measuring. For a single writer it takes on average 0.02s per 100 events appended (4,929 events/sec) and for 50 concurrent writers it's 50 x 100 events in 0.58s (8,586 events/sec).

For reading events it takes a single reader 3.67ms to read 100 events (27,211 events/sec) and for 50 concurrent readers it takes 0.19s (26,561 events/sec).

The purpose of the benchmark suite is to measure the performance impact of proposed changes, as opposed to looking at the raw numbers. The above figures are taken when run against a local PostgreSQL database. You can run the benchmarks against your own hardware to get indicative performance figures for the Event Store.

The benchmark suite is configured to use Erlang'sexternal term format serialization. Using another serialization format, such as JSON, will likely have a negative impact on performance.

Tests can be run using any Postgres database instance, including via Docker.

To use Docker, first pull the latest Postgres image:

docker pull postgres

Atmpfs mount can be used to run the Docker container with the Postgres data directory stored in memory.

docker run --rm \  --name postgres \  --tmpfs=/pgtmpfs \  -e PGDATA=/pgtmpfs \  -e POSTGRES_PASSWORD=postgres \  -e POSTGRES_USER=postgres \  -p 5432:5432 \  postgres

Alternatively, use Docker Compose

docker compose up

Create and initialize the test event store databases:

MIX_ENV=test mix event_store.setup

Run the test suite:

mix test

Pull requests to contribute new or improved features, and extend documentation are most welcome.

Please follow the existing coding conventions, or refer to theElixir style guide.

You should include unit tests to cover any changes.

EventStore exists thanks to the following people who have contributed.

• Andreas Riemer
• Andrey Akulov
• Basile Nouvellet
• Ben Smith
• Bruce Williams
• Chris Brodt
• Chris Martin
• Christian Green
• Craig Savolainen
• Damir Vandic
• David Soff
• Derek Kraan
• Diogo Scudelletti
• Dominik Guzei
• Douglas Vought
• Eamon Taaffe
• Floris Huetink
• Fredrik Teschke
• Ilya Suzdalnitskiy
• Jan Vereecken
• Kai Kuchenbecker
• Kaz Walker
• Morten Berg Nissen
• Nicholas Henry
• Olafur Arason
• Ole Michaelis
• Paul Iannazzo
• Piotr Szmielew
• Raphaël Lustin
• Ryan Young
• Samuel Roze
• Simon Harris
• Stuart Corbishley
• Thomas Coopman
• Victor Oliveira Nascimento
• Yamil Díaz Aguirre
• Yannis Weishaupt
• Yordis Prieto

Pleaseopen an issue if you encounter a problem, or need assistance. You can also seek help in the #commanded channel in theofficial Elixir Slack.