1,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

1,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

&MtmUseDtm,

true,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

&MtmPreserveCommitOrder,

true,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

&MtmVolksWagenMode,

false,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

1,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

1,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

0,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

1,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

1024*1024,

INT_MAX,

PGC_BACKEND,

0,

GUC_NO_SHOW_ALL,

NULL,

NULL,

if ( (*(int*)i)!=MtmNodeId-1)

return;

stripped_notice=palloc0(len);

stripped_notice=palloc0(len+1);

if (*notice=='N')

{

}

{

stripped_notice=notice;

strncpy(stripped_notice,notice,len+1);

MTM_ELOG(WARNING,"%s",stripped_notice);

}

<programlisting>

shared_preload_libraries = 'multimaster'

</programlisting>

<tip>

<para>If the <varname>shared_preload_libaries</varname> variable is already defined in <filename>postgresql.auto.conf</filename>, you will need to modify its value using the <literal>ALTER SYSTEM</literal> command. For details, see <xref linkend="config-setting-configuration-file">.

</para>

</tip>

</listitem>

<listitem><para>Specify the transaction isolation level for your cluster. <filename>multimaster</filename> currently supports <link linkend="xact-read-committed">read committed</link> and <link linkend="xact-repeatable-read">repeatable read</link> isolation levels.

<programlisting>

# of connection strings

# to neighbor nodes

</programlisting>

<para>The <literal>max_nodes</literal> variable defines the cluster size. In most cases, three cluster nodes are enough to ensure high availability. Since the data on all cluster nodes is the same, you do not typically need more than five cluster nodes.</para>

<para>The <literal>multimaster.max_nodes</literal> variable defines the maximum cluster size. If you plan to add new nodes to your cluster, the <literal>multimaster.max_nodes</literal> value should exceed the initial number of nodes. In this case, you can add new nodes without restarting <productname>&productname;</productname>.

</para>

<para>In most cases, three cluster nodes are enough to ensure high availability. Since the data on all cluster nodes is the same, you do not typically need more than five cluster nodes.

</para>

<para>If you would like to change the default connection settings for a cluster node, you can add other <link linkend="libpq-paramkeywords"> connection parameters</link> to the corresponding connection string in the <varname>multimaster.conn_strings</varname> variable. Additionally, you can specify the <literal>arbiter_process</literal> parameter to change the port used by the arbiter process to connect to the nodes. For details, see <xref linkend="multimaster-conn-strings"> and <xref linkend="multimaster-arbiter-port">.</para>

<important><para>The

<literal>node_id</literal> variable takes natural

<literal>multimaster.node_id</literal> variable takes natural

numbers starting from 1, without any gaps in numbering.

For example, for a cluster of five nodes, set node IDs

to 1, 2, 3, 4, and 5. In the

<literal>conn_strings</literal> variable, make sure to

<literal>multimaster.conn_strings</literal> variable, make sure to

list the nodes in the order of their IDs. The

<literal>conn_strings</literal> variable must be the

<literal>multimaster.conn_strings</literal> variable must be the

same on all nodes.</para></important>

</listitem>

following variables:

</para>

<itemizedlist>

<listitem>

<para>

<varname>multimaster.max_recovery_lag</varname> &mdash; sets the

maximum size of WAL. Upon reaching the

<varname>multimaster.max_recovery_lag</varname> threshold,

WAL for the disconnected node is overwritten. At this

point, automatic recovery is no longer possible. In this case, you can <link linkend="multimaster-restoring-a-node-manually">restore the node manually</link> by cloning the data from one of the alive nodes using <application>pg_basebackup</application>.

</para>

</listitem>

<listitem>

<para>

<varname>multimaster.min_recovery_lag</varname> &mdash; sets the

minimal WAL lag between the node to be restored and the current cluster state. When the

disconnected node is fast-forwarded up to the

minimal WAL lag between the node to be restored and the current cluster state.

By default, <varname>multimaster.min_recovery_lag</varname> is set to 100KB.

When the disconnected node is fast-forwarded up to the

<varname>multimaster.min_recovery_lag</varname> threshold,

<filename>multimaster</filename> stops all new commits to the

alive nodes until the node fully catches up with the

the cluster resumes its work.

</para>

</listitem>

<listitem>

<para>

<varname>multimaster.max_recovery_lag</varname> &mdash; sets the

maximum size of WAL. Upon reaching the

<varname>multimaster.max_recovery_lag</varname> threshold,

WAL for the disconnected node is overwritten. At this

point, automatic recovery is no longer possible. In this case, you can <link linkend="multimaster-restoring-a-node-manually">restore the node manually</link> by cloning the data from one of the alive nodes using <application>pg_basebackup</application>.

</para>

</listitem>

</itemizedlist>

<para>

By default, <varname>multimaster.max_recovery_lag</varname> is

set to1GB. Setting

set to100MB. Setting

<varname>multimaster.max_recovery_lag</varname> to a larger

value increases the timeframe for automatic recovery, but

requires more disk space for WAL collection.

<programlisting>

multimaster.max_nodes = 4

multimaster.node_id = 4

multimaster.conn_strings = 'dbname=mydb user=myuser host=node1, dbname=mydb user=myuser host=node2, dbname=mydb user=myuser host=node3, dbname=mydb user=myuser host=node4'

multimaster.conn_strings = 'dbname=mydb user=myuser host=node1, dbname=mydb user=myuser host=node2,

dbname=mydb user=myuser host=node3, dbname=mydb user=myuser host=node4'

</programlisting>

</listitem>

<listitem>

<orderedlist>

<listitem>

<para>

Change <literal>multimaster.conn_strings</literal> and

<literal>multimaster.max_nodes</literal> to include the new node.

Change <literal>multimaster.conn_strings</literal> and <literal>multimaster.max_nodes</literal> to include the new node.

</para>

</listitem>

<listitem>

replication to the new node.

</para>

</listitem>

<listitem>

<para>

Restart all cluster nodes.

</para>

</listitem>

</orderedlist>

<para>

<emphasis role="strong">See Also</emphasis></para>

For example, for a cluster of five nodes, set node IDs to 1, 2, 3,

4, and 5.

</para></listitem></varlistentry>

<varlistentry><term><varname>multimaster.conn_strings</varname><indexterm><primary><varname>multimaster.conn_strings</varname></primary></indexterm></term><listitem><para>

Connection strings for

connection string must include the name of the database to replicate

<varlistentry id="multimaster-conn-strings"><term><varname>multimaster.conn_strings</varname><indexterm><primary><varname>multimaster.conn_strings</varname></primary></indexterm></term><listitem><para>

Connection strings for each node of a multi-master cluster, separated by commas.

Eachconnection string must include the name of the database to replicate

and the cluster node domain name. For example, 'dbname=mydb

host=node1, dbname=mydb host=node2, dbname=mydb host=node3'.

Optionally, you can add other <link linkend="libpq-paramkeywords">connection parameters</link> to change the default connection settings.

Connection strings must appear in the order of the node IDs

specified in the <varname>multimaster.node_id</varname> variable.

Connection string for the i-th node must be on the i-th position.

This parameter must be identical on all nodes. You can also specify a

custom port for all connection strings using the

<varname>multimaster.arbiter_port</varname> variable.

You can also specify a

custom port for the arbiter to listen on using the <literal>arbiter_port</literal>

parameter. The provided value must be the same as the <varname>multimaster.arbiter_port</varname>

value specified for this node.

</para></listitem></varlistentry>

<varlistentry><term><varname>multimaster.max_nodes</varname><indexterm><primary><varname>multimaster.max_nodes</varname></primary></indexterm></term><listitem><para>

The maximum number of nodes allowed in the cluster. In most cases, three cluster nodes are enough to ensure high availability. Since the data on all cluster nodes is the same, you do not typically need more than five cluster nodes. The maximum possible number of nodes is limited to 64.</para><para>Default: the number of nodes specified in the <varname>multimaster.conn_strings</varname> variable

The maximum number of nodes allowed in the cluster. If you plan to add new nodes to your cluster, the <literal>multimaster.max_nodes</literal> value should exceed the initial number of nodes. In this case, you can add new nodes without restarting <productname>&productname;</productname>. In most cases, three cluster nodes are enough to ensure high availability. Since the data on all cluster nodes is the same, you do not typically need more than five cluster nodes. The maximum possible number of nodes is limited to 64.</para>

<para>Default: the number of nodes specified in the <varname>multimaster.conn_strings</varname> variable

</para></listitem></varlistentry>

<varlistentry><term><varname>multimaster.arbiter_port</varname><indexterm><primary><varname>multimaster.arbiter_port</varname></primary></indexterm></term><listitem><para>

Port for the arbiter

process to listen on. </para><para>Default: 5433

<varlistentry id="multimaster-arbiter-port">

<term><varname>multimaster.arbiter_port</varname><indexterm><primary><varname>multimaster.arbiter_port</varname></primary></indexterm></term><listitem><para>

Port for the arbiter process to listen on. If you change the default value, you must specify this value in the <literal>arbiter_port</literal>

parameter in the connection string of the corresponding node.</para>

<para>Default: 5433

</para></listitem></varlistentry>

<varlistentry><term><varname>multimaster.heartbeat_send_timeout</varname><indexterm><primary><varname>multimaster.heartbeat_send_timeout</varname></primary></indexterm></term><listitem><para>

Time interval

the data from one of the alive nodes using

<application>pg_basebackup</application> or a similar tool. If you set this

variable to zero, replication slot will not be dropped. </para><para>Default:

</para></listitem></varlistentry>

<varlistentry><term><varname>multimaster.ignore_tables_without_pk</varname><indexterm><primary><varname>multimaster.ignore_tables_without_pk</varname></primary></indexterm></term><listitem><para>

Boolean.

you set this variable, <filename>multimaster</filename> checks that

the cluster name is the same for all the cluster nodes.

</para></listitem></varlistentry>

<varlistentry>

<term><varname>multimaster.break_connection</varname>

<indexterm><primary><varname>multimaster.break_connection</varname></primary>

</indexterm>

</term>

<listitem>

<para>Break connection with clients connected to the node if this node disconnects

from the cluster. If this variable is set to <literal>false</literal>, the client stays

connected to the node but receives an error that the node is in minority.

</para>

<para>Default: <literal>false</literal>

</para>

</listitem>

</varlistentry>

<varlistentry>

<term><varname>multimaster.major_node</varname>

<indexterm><primary><varname>multimaster.major_node</varname></primary>

</indexterm>

</term>

<listitem>

<para>Node with this flag continues working even if it cannot access the majority of other nodes.

This is needed to break the symmetry if there is an even number of alive nodes in the cluster.

For example, a cluster with three nodes continues working if one of the nodes has crashed.

If connection between the remaining nodes is lost, the node with <varname>multimaster.major_node</varname> = <literal>true</literal> will continue working.

</para>

<important>

<para>This parameter should be used with caution. Only one node in the cluster

can have this parameter set to true. When set to <literal>true</literal> on several

nodes, this parameter can cause the split-brain problem.

</para>

</important>

</listitem>

</varlistentry>

<varlistentry>

<term><varname>multimaster.max_workers</varname>

<indexterm><primary><varname>multimaster.max_workers</varname></primary>

</indexterm>

</term>

<listitem>

<para>The maximum number of <literal>walreceiver</literal> workers on this server.

</para>

<important>

<para>This parameter should be used with caution. If the number of simultaneous transactions

in the whole cluster is bigger than the provided value, it can lead to undetected deadlocks.

</para>

</important>

</listitem>

</varlistentry>

<varlistentry>

<term><varname>multimaster.trans_spill_threshold</varname>

<indexterm><primary><varname>multimaster.trans_spill_threshold</varname></primary>

</indexterm>

</term>

<listitem>

<para>The maximal size of transaction, in MB. When this threshold is reached, the transaction is written to the disk.

</para>

<para>Default: 100

</para>

</listitem>

</varlistentry>

</variablelist>

</sect3>

<sect3 id="multimaster-functions"><title>Functions</title>