- Notifications
You must be signed in to change notification settings - Fork28
postgrespro/postgres_cluster
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
This repo implements distributed transaction manager using Snapshot Sharing mechanism. General concepts and alternative approaches described in postgres wikihttps://wiki.postgresql.org/wiki/DTM.
Backend-DTM protocol description can be found indtmd/README.
- Patch postgres using xtm.patch. After that build and install postgres in usual way.
cd~/code/postgrespatch -p1<~/code/pg_dtm/xtm.patch
- Install pg_dtm extension.
export PATH=/path/to/pgsql/bin/:$PATHcd~/code/pg_dtmmake&& make install
- Run dtmd.
cd~/code/pg_dtm/dtmdmakemkdir /tmp/clog./bin/dtmd&
- To run something meaningful you need at leat two postgres instances. Also pg_dtm requires presense in
shared_preload_libraries.
initdb -D ./install/data1initdb -D ./install/data2echo"port = 5433">> ./install/data2/postgresql.confecho"shared_preload_libraries = 'pg_dtm'">> ./install/data1/postgresql.confecho"shared_preload_libraries = 'pg_dtm'">> ./install/data2/postgresql.confpg_ctl -D ./install/data1 -l ./install/data1/log startpg_ctl -D ./install/data2 -l ./install/data2/log start
For a cluster-wide deploy we use ansible, more details in tests/deploy_layouts. (Ansible instructions will be later)
Now cluster is running and you can use global tx between two nodes. Let's connect to postgres instances at different ports:
create extension pg_dtm;-- node1createtableaccounts(user_idint, amountint);-- node1insert into accounts (select2*generate_series(1,100)-1,0);-- node1, odd user_id's create extension pg_dtm;-- node2createtableaccounts(user_idint, amountint);-- node2insert into accounts (select2*generate_series(1,100),0);-- node2, even user_id'sselect dtm_begin_transaction();-- node1, returns global xid, e.g. 42select dtm_join_transaction(42);-- node2, join global txbegin;-- node1begin;-- node2update accountsset amount=amount-100where user_id=1;-- node1, transfer money from user#1update accountsset amount=amount+100where user_id=2;-- node2, to user#2commit;-- node1, blocks until second commit happendcommit;-- node2
To ensure consistency we use simple bank test: perform a lot of simultaneous transfers between accounts on different servers, while constantly checking total amount of money on all accounts. This test can be found in tests/perf.
> go run ./tests/perf/* -C value Connection string (repeatfor multiple connections) -a int The number of bank accounts (default 100000) -b string Backend to use. Possible optinos: transfers, fdw, pgshard, readers. (default"transfers") -gUse DTM to keep global consistency -iInit database -lUse'repeatable read' isolation level instead of'read committed' -n int The number updates each writer (readerincase of Reades backend) performs (default 10000) -pUse parallel execs -r int The number of readers (default 1) -s int StartID. Script will update rows starting from this value -vShow progress and other stufffor mortals -w int The number of writers (default 8)
So previous installation can be initialized with:
go run ./tests/perf/*.go \-C "dbname=postgres port=5432" \-C "dbname=postgres port=5433" \-g -iand tested with:
go run ./tests/perf/*.go \-C "dbname=postgres port=5432" \-C "dbname=postgres port=5433" \-gWe also provide a patch, that enables support of global transactions with postres_fdw. After patching and installing postres_fdw it is possible to run same test via fdw usig key-b fdw.
Citus Data have branch in their pg_shard repo, that interacts with transaction manager.https://github.com/citusdata/pg_shard/tree/transaction_manager_integrationTo use this feature one should have following line in postgresql.conf (or set it via GUC)
pg_shard.use_dtm_transactions = 1About
Various experiments with PostgreSQL clustering
Resources
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Releases
Packages0
Uh oh!
There was an error while loading.Please reload this page.