Multi-master isanextensionandset of patchesto a Postegres database,that turnsPostgres into a synchronous shared-nothing cluster to provide OLTP scalability and high availability with automatic disaster recovery.

`multimaster` isa PostgreSQLextensionwith aset of patches that turnsPostgreSQL into a synchronous shared-nothing cluster to provideOnline Transaction Processing (OLTP) scalability and high availability with automatic disaster recovery.

Multi-masterreplicates same database to all nodesincluster and allowswrites toeach node.Transaction isolation is enforced cluster-wide, soin case of concurrent updates on different nodes database will use the same conflict resolution rules (mvcc withrepeatable read isolation level) as single node uses for concurrent backends and always stays in consistent state. Anywriting transactionwill writeto all nodes,hence increasing commit latency foramount oftimeproportional to roundtrip between nodes neddedfor synchronization. Readonly transactions and queries executed locally without measurable overhead.Replication mechanism itself based on logical decoding and earlier version of pglogical extension provided for community by 2ndQuadrant team.

The`multimaster` extensionreplicatesthesame database to all nodesof thecluster and allowswrite transactions oneach node.To ensure data consistencyinthecase of concurrent updates,`multimaster` enforces transaction isolation cluster-wide, using multiversion concurrency control (MVCC) at therepeatable read isolation level. Anywrite transactionis synchronously replicatedto all nodes,which increases commit latency forthetimerequiredfor synchronization. Read-only transactions and queriesareexecuted locally, withoutanymeasurable overhead.

Cluster consistingofN nodes can continue to work while majority of initial nodes are alive and reachable by other nodes. This is done by using 3 phase commit protocolandheartbeats for failure discovery. Node thatisbrought backto clustercanbe fast-forwadedto actual stateautomatically in case whentransactions logstill exists sincethe time when node was excluded from cluster (that is configurable).

To ensure high availability and fault toleranceofthe cluster,`multimaster` uses three-phase commit protocol and heartbeats for failure discovery. A multi-master cluster of N nodes can continue working while the majority of the nodes are aliveandreachable by other nodes. When the nodeisreconnectedtothecluster,`multimaster`canautomatically fast-forward the nodetotheactual statebased on thetransactions log(WAL). If WAL is no longer available forthe time whenthenode was excluded fromthecluster, you can restore the node using`pg_basebackup`.

For details on the`multimaster` internals, see the[Architecture](/contrib/mmts/doc/architecture.md) page.

1.[Administration](doc/administration.md)

1.[Installation](doc/administration.md#installation)

1.[Setting up empty cluster](doc/administration.md#setting-up-empty-cluster)

1.[Setting up cluster from pre-existing database](doc/administration.md#setting-up-cluster-from-pre-existing-database)

1.[Tuning configuration params](doc/administration.md#tuning-configuration-params)

1.[Monitoring](doc/administration.md#monitoring)

1.[Adding nodes to cluster](doc/administration.md#adding-nodes-to-cluster)

1.[Excluding nodes from cluster](doc/administration.md#excluding-nodes-from-cluster)

1.[Architecture and internals](doc/architecture.md)

1.[List of configuration variables](doc/configuration.md)

1.[Built-in functions and views](doc/configuration.md)

1.[Setting up a Multi-Master Cluster](doc/administration.md#setting-up-a-multi-master-cluster)

1.[Tuning configuration params](doc/administration.md#tuning-configuration-parameters)

1.[Monitoring Cluster Status](doc/administration.md#monitoring-cluster-status)

1.[Adding New Nodes to the Cluster](doc/administration.md#adding-new-nodes-to-the-cluster)

1.[Excluding Nodes from the Cluster](doc/administration.md#excluding-nodes-from-the-cluster)

1.[Architecture](doc/architecture.md)

1.[Configuration Variables](doc/configuration.md)

1.[Built-in Functions and Views](doc/functions.md)

* Commit latency.

Current implementation of logical replication sends data to subscriber nodes only after local commit, so in case of heavy-write transaction user will wait for transaction processing two times: on local node and on all other nodes (simultaneosly). We have plans to address this issue in future.

*`multimaster` can only replicate one database per cluster.

* The replicated tables must have primary keys. Otherwise,`multimaster` cannot perform logical replication.

* DDL replication.

While data is replicated on logical level, DDL replicated by statements performing distributed commit with the same statement. Some complex DDL scenarious including stored procedures and temp temp tables can work differently comparing to standalone postgres. We are working right now on proving full compatibility with ordinary postgres. Currently we are passing 162 of 166 postgres regression tests.

While`multimaster` replicates data on the logical level, DDL is replicated on the statement level, which causes distributed commits of the same statement on different nodes. As a result, complex DDL scenarios, such as stored procedures and temporary tables, may work differently as compared to the standard PostgreSQL.

* Commit latency.

The current implementation of logical replication sends data to subscriber nodes only after the local commit. In case of a heavy-write transaction, you have to wait for transaction processing twice: on the local node and on all the other nodes (simultaneously).

* Isolation level.

Multimaster currently support only_repeatable__read_ isolation level. This is stricter than default_read__commited_, but also increases probability of serialization failure during commit._Serializable_ level isn't supported yet.

The`multimaster` extenstion currently supports only the_repeatable__read_ isolation level. This is stricter than the default_read__commited_ level, but also increases probability of serialization failure during commit._Serializable_ level is not supported yet.

* One database per cluster.

The`multimaster` extension currently passes 162 of 166 postgres regression tests. We are working right now on proving full compatibility with the standard PostgreSQL.

Multi-master developed by the PostgresPro team.

The replication mechanism is based on logical decoding and an earlier version of the`pglogical` extension provided for community by the 2ndQuadrant team.