Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitf49a80c

Browse files
alvherreArseny Sher
and
Arseny Sher
committed
Fix "base" snapshot handling in logical decoding
Two closely related bugs are fixed. First, xmin of logical slots wasadvanced too early. During xl_running_xacts processing, xmin of theslot was set to the oldest running xid in the record, but that's wrong:actually, snapshots which will be used for not-yet-replayed transactionsmight consider older txns as running too, so we need to keep xmin backfor them. The problem wasn't noticed earlier because DDL which allowsto delete tuple (set xmax) while some another not-yet-committedtransaction looks at it is pretty rare, if not unique: e.g. all forms ofALTER TABLE which change schema acquire ACCESS EXCLUSIVE lockconflicting with any inserts. The included test case (test_decoding'soldest_xmin) uses ALTER of a composite type, which doesn't have suchinterlocking.To deal with this, we must be able to quickly retrieve oldest xmin(oldest running xid among all assigned snapshots) from ReorderBuffer. Tofix, add another list of ReorderBufferTXNs to the reorderbuffer, wheretransactions are sorted by base-snapshot-LSN. This is slightlydifferent from the existing (sorted by first-LSN) list, because atransaction can have an earlier LSN but a later Xmin, if its firstrecord does not obtain an xmin (eg. xl_xact_assignment). Note this newlist doesn't fully replace the existing txn list: we still need that oneto prevent WAL recycling.The second issue concerns SnapBuilder snapshots and subtransactions.SnapBuildDistributeNewCatalogSnapshot never assigned a snapshot to atransaction that is known to be a subtxn, which is good in the commoncase that the top-level transaction already has one (no point in doingso), but a bug otherwise. To fix, arrange to transfer the snapshot fromthe subtxn to its top-level txn as soon as the kinship gets known.test_decoding's snapshot_transfer verifies this.Also, fix a minor memory leak: refcount of toplevel's old base snapshotwas not decremented when the snapshot is transferred from child.Liberally sprinkle code comments, and rewrite a few existing ones. Thispart is my (Álvaro's) contribution to this commit, as I had to write allthose comments in order to understand the existing code and Arseny'spatch.Reported-by: Arseny Sher <a.sher@postgrespro.ru>Diagnosed-by: Arseny Sher <a.sher@postgrespro.ru>Co-authored-by: Arseny Sher <a.sher@postgrespro.ru>Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>Reviewed-by: Antonin Houska <ah@cybertec.at>Discussion:https://postgr.es/m/87lgdyz1wj.fsf@ars-thinkpad
1 parent4d54543 commitf49a80c

File tree

8 files changed

+409
-104
lines changed

8 files changed

+409
-104
lines changed

‎contrib/test_decoding/Makefile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,8 @@ regresscheck-install-force: | submake-regress submake-test_decoding temp-install
5050
$(pg_regress_installcheck)\
5151
$(REGRESSCHECKS)
5252

53-
ISOLATIONCHECKS=mxact delayed_startup ondisk_startup concurrent_ddl_dml
53+
ISOLATIONCHECKS=mxact delayed_startup ondisk_startup concurrent_ddl_dml\
54+
oldest_xmin snapshot_transfer
5455

5556
isolationcheck: | submake-isolation submake-test_decoding temp-install
5657
$(pg_isolation_regress_check)\
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
Parsed test spec with 2 sessions
2+
3+
starting permutation: s0_begin s0_getxid s1_begin s1_insert s0_alter s0_commit s0_checkpoint s0_get_changes s1_commit s0_vacuum s0_get_changes
4+
step s0_begin: BEGIN;
5+
step s0_getxid: SELECT txid_current() IS NULL;
6+
?column?
7+
8+
f
9+
step s1_begin: BEGIN;
10+
step s1_insert: INSERT INTO harvest VALUES ((1, 2, 3));
11+
step s0_alter: ALTER TYPE basket DROP ATTRIBUTE mangos;
12+
step s0_commit: COMMIT;
13+
step s0_checkpoint: CHECKPOINT;
14+
step s0_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
15+
data
16+
17+
step s1_commit: COMMIT;
18+
step s0_vacuum: VACUUM FULL;
19+
step s0_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
20+
data
21+
22+
BEGIN
23+
table public.harvest: INSERT: fruits[basket]:'(1,2,3)'
24+
COMMIT
25+
?column?
26+
27+
stop
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
Parsed test spec with 2 sessions
2+
3+
starting permutation: s0_begin s0_begin_sub0 s0_log_assignment s0_sub_get_base_snap s1_produce_new_snap s0_insert s0_end_sub0 s0_commit s0_get_changes
4+
step s0_begin: BEGIN;
5+
step s0_begin_sub0: SAVEPOINT s0;
6+
step s0_log_assignment: SELECT txid_current() IS NULL;
7+
?column?
8+
9+
f
10+
step s0_sub_get_base_snap: INSERT INTO dummy VALUES (0);
11+
step s1_produce_new_snap: ALTER TABLE harvest ADD COLUMN mangos int;
12+
step s0_insert: INSERT INTO harvest VALUES (1, 2, 3);
13+
step s0_end_sub0: RELEASE SAVEPOINT s0;
14+
step s0_commit: COMMIT;
15+
step s0_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
16+
data
17+
18+
BEGIN
19+
table public.dummy: INSERT: i[integer]:0
20+
table public.harvest: INSERT: apples[integer]:1 pears[integer]:2 mangos[integer]:3
21+
COMMIT
22+
?column?
23+
24+
stop
25+
26+
starting permutation: s0_begin s0_begin_sub0 s0_log_assignment s0_begin_sub1 s0_sub_get_base_snap s1_produce_new_snap s0_insert s0_end_sub1 s0_end_sub0 s0_commit s0_get_changes
27+
step s0_begin: BEGIN;
28+
step s0_begin_sub0: SAVEPOINT s0;
29+
step s0_log_assignment: SELECT txid_current() IS NULL;
30+
?column?
31+
32+
f
33+
step s0_begin_sub1: SAVEPOINT s1;
34+
step s0_sub_get_base_snap: INSERT INTO dummy VALUES (0);
35+
step s1_produce_new_snap: ALTER TABLE harvest ADD COLUMN mangos int;
36+
step s0_insert: INSERT INTO harvest VALUES (1, 2, 3);
37+
step s0_end_sub1: RELEASE SAVEPOINT s1;
38+
step s0_end_sub0: RELEASE SAVEPOINT s0;
39+
step s0_commit: COMMIT;
40+
step s0_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
41+
data
42+
43+
BEGIN
44+
table public.dummy: INSERT: i[integer]:0
45+
table public.harvest: INSERT: apples[integer]:1 pears[integer]:2 mangos[integer]:3
46+
COMMIT
47+
?column?
48+
49+
stop
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Test advancement of the slot's oldest xmin
2+
3+
setup
4+
{
5+
SELECT'init'FROMpg_create_logical_replication_slot('isolation_slot','test_decoding');--mustbefirstwriteinxact
6+
DROPTYPEIFEXISTSbasket;
7+
CREATETYPEbasketAS (applesinteger,pearsinteger,mangosinteger);
8+
DROPTABLEIFEXISTSharvest;
9+
CREATETABLEharvest(fruitsbasket);
10+
}
11+
12+
teardown
13+
{
14+
DROPTABLEIFEXISTSharvest;
15+
DROPTYPEIFEXISTSbasket;
16+
SELECT'stop'FROMpg_drop_replication_slot('isolation_slot');
17+
}
18+
19+
session"s0"
20+
step"s0_begin" {BEGIN; }
21+
step"s0_getxid" {SELECTtxid_current()ISNULL; }
22+
step"s0_alter" {ALTERTYPEbasketDROPATTRIBUTEmangos; }
23+
step"s0_commit" {COMMIT; }
24+
step"s0_checkpoint" {CHECKPOINT; }
25+
step"s0_vacuum" {VACUUMFULL; }
26+
step"s0_get_changes" {SELECTdataFROMpg_logical_slot_get_changes('isolation_slot',NULL,NULL,'include-xids','0','skip-empty-xacts','1'); }
27+
28+
session"s1"
29+
step"s1_begin" {BEGIN; }
30+
step"s1_insert" {INSERTINTOharvestVALUES ((1,2,3)); }
31+
step"s1_commit" {COMMIT; }
32+
33+
# Checkpoint with following get_changes forces to advance xmin. ALTER of a
34+
# composite type is a rare form of DDL which allows T1 to see the tuple which
35+
# will be removed (xmax set) before T1 commits. That is, interlocking doesn't
36+
# forbid modifying catalog after someone read it (and didn't commit yet).
37+
permutation"s0_begin""s0_getxid""s1_begin""s1_insert""s0_alter""s0_commit""s0_checkpoint""s0_get_changes""s1_commit""s0_vacuum""s0_get_changes"
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Test snapshot transfer from subxact to top-level and receival of later snaps.
2+
3+
setup
4+
{
5+
SELECT'init'FROMpg_create_logical_replication_slot('isolation_slot','test_decoding');--mustbefirstwriteinxact
6+
DROPTABLEIFEXISTSdummy;
7+
CREATETABLEdummy(iint);
8+
DROPTABLEIFEXISTSharvest;
9+
CREATETABLEharvest(applesint,pearsint);
10+
}
11+
12+
teardown
13+
{
14+
DROPTABLEIFEXISTSharvest;
15+
DROPTABLEIFEXISTSdummy;
16+
SELECT'stop'FROMpg_drop_replication_slot('isolation_slot');
17+
}
18+
19+
session"s0"
20+
step"s0_begin" {BEGIN; }
21+
step"s0_begin_sub0" {SAVEPOINTs0; }
22+
step"s0_log_assignment" {SELECTtxid_current()ISNULL; }
23+
step"s0_begin_sub1" {SAVEPOINTs1; }
24+
step"s0_sub_get_base_snap" {INSERTINTOdummyVALUES (0); }
25+
step"s0_insert" {INSERTINTOharvestVALUES (1,2,3); }
26+
step"s0_end_sub0" {RELEASESAVEPOINTs0; }
27+
step"s0_end_sub1" {RELEASESAVEPOINTs1; }
28+
step"s0_insert2" {INSERTINTOharvestVALUES (1,2,3,4); }
29+
step"s0_commit" {COMMIT; }
30+
step"s0_get_changes" {SELECTdataFROMpg_logical_slot_get_changes('isolation_slot',NULL,NULL,'include-xids','0','skip-empty-xacts','1'); }
31+
32+
session"s1"
33+
step"s1_produce_new_snap" {ALTERTABLEharvestADDCOLUMNmangosint; }
34+
35+
# start top-level without base snap, get base snap in subxact, then create new
36+
# snap and make sure it is queued.
37+
permutation"s0_begin""s0_begin_sub0""s0_log_assignment""s0_sub_get_base_snap""s1_produce_new_snap""s0_insert""s0_end_sub0""s0_commit""s0_get_changes"
38+
39+
# In previous test, we firstly associated subxact with xact and only then got
40+
# base snap; now nest one more subxact to get snap first and only then (at
41+
# commit) associate it with toplevel.
42+
permutation"s0_begin""s0_begin_sub0""s0_log_assignment""s0_begin_sub1""s0_sub_get_base_snap""s1_produce_new_snap""s0_insert""s0_end_sub1""s0_end_sub0""s0_commit""s0_get_changes"

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp