Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit387abe8

Browse files
committed
Fix pruning of locked and updated tuples.
Previously it was possible that a tuple was not pruned during vacuum,even though its update xmax (i.e. the updating xid in a multixact withboth key share lockers and an updater) was below the cutoff horizon.As the freezing code assumed, rightly so, that that's not supposed tohappen, xmax would be preserved (as a member of a new multixact orxmax directly). That causes two problems: For one the tuple is belowthe xmin horizon, which can cause problems if the clog is truncated oronce there's an xid wraparound. The bigger problem is that that willbreak HOT chains, which in turn can lead two to breakages: First,failing index lookups, which in turn can e.g lead to constraints beingviolated. Second, future hot prunes / vacuums can end up makinginvisible tuples visible again. There's other harmful scenarios.Fix the problem by recognizing that tuples can be DEAD instead ofRECENTLY_DEAD, even if the multixactid has alive members, if theupdate_xid is below the xmin horizon. That's safe because newerversions of the tuple will contain the locking xids.A followup commit will harden the code somewhat against future similarbugs and already corrupted data.Author: Andres Freund, with changes by Alvaro HerreraReported-By: Daniel WoodAnalyzed-By: Andres Freund, Alvaro Herrera, Robert Haas, Peter Geoghegan, Daniel Wood, Yi Wen Wong, Michael PaquierReviewed-By: Alvaro Herrera, Robert Haas, Michael PaquierDiscussion:https://postgr.es/m/E5711E62-8FDF-4DCA-A888-C200BF6B5742@amazon.comhttps://postgr.es/m/20171102112019.33wb7g5wp4zpjelu@alap3.anarazel.deBackpatch: 9.3-
1 parent152a569 commit387abe8

File tree

4 files changed

+80
-121
lines changed

4 files changed

+80
-121
lines changed

‎src/backend/utils/time/tqual.c

Lines changed: 24 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1434,49 +1434,40 @@ HeapTupleSatisfiesVacuum(HeapTupleHeader tuple, TransactionId OldestXmin,
14341434

14351435
if (tuple->t_infomask&HEAP_XMAX_IS_MULTI)
14361436
{
1437-
TransactionIdxmax;
1438-
1439-
if (MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple)))
1440-
{
1441-
/* already checked above */
1442-
Assert(!HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask));
1443-
1444-
xmax=HeapTupleGetUpdateXid(tuple);
1437+
TransactionIdxmax=HeapTupleGetUpdateXid(tuple);
14451438

1446-
/* not LOCKED_ONLY, so it has to have an xmax */
1447-
Assert(TransactionIdIsValid(xmax));
1448-
1449-
if (TransactionIdIsInProgress(xmax))
1450-
returnHEAPTUPLE_DELETE_IN_PROGRESS;
1451-
elseif (TransactionIdDidCommit(xmax))
1452-
/* there are still lockers around -- can't return DEAD here */
1453-
returnHEAPTUPLE_RECENTLY_DEAD;
1454-
/* updating transaction aborted */
1455-
returnHEAPTUPLE_LIVE;
1456-
}
1457-
1458-
Assert(!(tuple->t_infomask&HEAP_XMAX_COMMITTED));
1459-
1460-
xmax=HeapTupleGetUpdateXid(tuple);
1439+
/* already checked above */
1440+
Assert(!HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask));
14611441

14621442
/* not LOCKED_ONLY, so it has to have an xmax */
14631443
Assert(TransactionIdIsValid(xmax));
14641444

1465-
/* multi is not running -- updating xact cannot be */
1466-
Assert(!TransactionIdIsInProgress(xmax));
1467-
if (TransactionIdDidCommit(xmax))
1445+
if (TransactionIdIsInProgress(xmax))
1446+
returnHEAPTUPLE_DELETE_IN_PROGRESS;
1447+
elseif (TransactionIdDidCommit(xmax))
14681448
{
1449+
/*
1450+
* The multixact might still be running due to lockers. If the
1451+
* updater is below the xid horizon, we have to return DEAD
1452+
* regardless -- otherwise we could end up with a tuple where the
1453+
* updater has to be removed due to the horizon, but is not pruned
1454+
* away. It's not a problem to prune that tuple, because any
1455+
* remaining lockers will also be present in newer tuple versions.
1456+
*/
14691457
if (!TransactionIdPrecedes(xmax,OldestXmin))
14701458
returnHEAPTUPLE_RECENTLY_DEAD;
1471-
else
1472-
returnHEAPTUPLE_DEAD;
1459+
1460+
returnHEAPTUPLE_DEAD;
1461+
}
1462+
elseif (!MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple)))
1463+
{
1464+
/*
1465+
* Not in Progress, Not Committed, so either Aborted or crashed.
1466+
* Mark the Xmax as invalid.
1467+
*/
1468+
SetHintBits(tuple,buffer,HEAP_XMAX_INVALID,InvalidTransactionId);
14731469
}
14741470

1475-
/*
1476-
* Not in Progress, Not Committed, so either Aborted or crashed.
1477-
* Remove the Xmax.
1478-
*/
1479-
SetHintBits(tuple,buffer,HEAP_XMAX_INVALID,InvalidTransactionId);
14801471
returnHEAPTUPLE_LIVE;
14811472
}
14821473

Lines changed: 21 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -1,101 +1,36 @@
1-
Parsed test spec with2 sessions
1+
Parsed test spec with3 sessions
22

3-
starting permutation: s1_update s1_commit s1_vacuum s2_key_share s2_commit
3+
starting permutation: s1_begin s2_begin s3_begin s1_update s2_key_share s3_key_share s1_update s1_commit s2_commit s2_vacuum s1_selectone s3_commit s2_vacuum s1_selectall
4+
step s1_begin: BEGIN;
5+
step s2_begin: BEGIN;
6+
step s3_begin: BEGIN;
47
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
5-
step s1_commit: COMMIT;
6-
step s1_vacuum: VACUUM FREEZE tab_freeze;
78
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
89
id
910

1011
3
11-
step s2_commit: COMMIT;
12-
13-
starting permutation: s1_update s1_commit s2_key_share s1_vacuum s2_commit
14-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
15-
step s1_commit: COMMIT;
16-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
12+
step s3_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
1713
id
1814

1915
3
20-
step s1_vacuum: VACUUM FREEZE tab_freeze;
21-
step s2_commit: COMMIT;
22-
23-
starting permutation: s1_update s1_commit s2_key_share s2_commit s1_vacuum
2416
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
2517
step s1_commit: COMMIT;
26-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
27-
id
28-
29-
3
3018
step s2_commit: COMMIT;
31-
step s1_vacuum: VACUUM FREEZE tab_freeze;
19+
step s2_vacuum: VACUUM FREEZE tab_freeze;
20+
step s1_selectone:
21+
BEGIN;
22+
SET LOCAL enable_seqscan = false;
23+
SET LOCAL enable_bitmapscan = false;
24+
SELECT * FROM tab_freeze WHERE id = 3;
25+
COMMIT;
3226

33-
starting permutation: s1_update s2_key_share s1_commit s1_vacuum s2_commit
34-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
35-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
36-
id
27+
id name x
3728

38-
3
39-
step s1_commit: COMMIT;
40-
step s1_vacuum: VACUUM FREEZE tab_freeze;
41-
step s2_commit: COMMIT;
42-
43-
starting permutation: s1_update s2_key_share s1_commit s2_commit s1_vacuum
44-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
45-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
46-
id
47-
48-
3
49-
step s1_commit: COMMIT;
50-
step s2_commit: COMMIT;
51-
step s1_vacuum: VACUUM FREEZE tab_freeze;
52-
53-
starting permutation: s1_update s2_key_share s2_commit s1_commit s1_vacuum
54-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
55-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
56-
id
29+
3 333 2
30+
step s3_commit: COMMIT;
31+
step s2_vacuum: VACUUM FREEZE tab_freeze;
32+
step s1_selectall: SELECT * FROM tab_freeze ORDER BY name, id;
33+
id name x
5734

58-
3
59-
step s2_commit: COMMIT;
60-
step s1_commit: COMMIT;
61-
step s1_vacuum: VACUUM FREEZE tab_freeze;
62-
63-
starting permutation: s2_key_share s1_update s1_commit s1_vacuum s2_commit
64-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
65-
id
66-
67-
3
68-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
69-
step s1_commit: COMMIT;
70-
step s1_vacuum: VACUUM FREEZE tab_freeze;
71-
step s2_commit: COMMIT;
72-
73-
starting permutation: s2_key_share s1_update s1_commit s2_commit s1_vacuum
74-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
75-
id
76-
77-
3
78-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
79-
step s1_commit: COMMIT;
80-
step s2_commit: COMMIT;
81-
step s1_vacuum: VACUUM FREEZE tab_freeze;
82-
83-
starting permutation: s2_key_share s1_update s2_commit s1_commit s1_vacuum
84-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
85-
id
86-
87-
3
88-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
89-
step s2_commit: COMMIT;
90-
step s1_commit: COMMIT;
91-
step s1_vacuum: VACUUM FREEZE tab_freeze;
92-
93-
starting permutation: s2_key_share s2_commit s1_update s1_commit s1_vacuum
94-
step s2_key_share: SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE;
95-
id
96-
97-
3
98-
step s2_commit: COMMIT;
99-
step s1_update: UPDATE tab_freeze SET x = x + 1 WHERE id = 3;
100-
step s1_commit: COMMIT;
101-
step s1_vacuum: VACUUM FREEZE tab_freeze;
35+
1 111 0
36+
3 333 2

‎src/test/isolation/isolation_schedule

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,5 +30,6 @@ test: lock-committed-keyupdate
3030
test: update-locked-tuple
3131
test: propagate-lock-delete
3232
test: tuplelock-conflict
33+
test: freeze-the-dead
3334
test: drop-index-concurrently-1
3435
test: timeouts

‎src/test/isolation/specs/freeze-the-dead.spec

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,44 @@ teardown
1616
}
1717

1818
session"s1"
19-
setup{BEGIN; }
19+
step"s1_begin"{BEGIN; }
2020
step"s1_update"{UPDATEtab_freezeSETx=x+1WHEREid=3; }
2121
step"s1_commit"{COMMIT; }
2222
step"s1_vacuum"{VACUUMFREEZEtab_freeze; }
23+
step"s1_selectone"{
24+
BEGIN;
25+
SETLOCALenable_seqscan=false;
26+
SETLOCALenable_bitmapscan=false;
27+
SELECT*FROMtab_freezeWHEREid=3;
28+
COMMIT;
29+
}
30+
step"s1_selectall"{SELECT*FROMtab_freezeORDERBYname,id; }
31+
step"s1_reindex"{REINDEXTABLEtab_freeze; }
2332

2433
session"s2"
25-
setup{BEGIN; }
34+
step"s2_begin"{BEGIN; }
2635
step"s2_key_share"{SELECTidFROMtab_freezeWHEREid=3FORKEYSHARE; }
2736
step"s2_commit"{COMMIT; }
37+
step"s2_vacuum"{VACUUMFREEZEtab_freeze; }
38+
39+
session"s3"
40+
step"s3_begin"{BEGIN; }
41+
step"s3_key_share"{SELECTidFROMtab_freezeWHEREid=3FORKEYSHARE; }
42+
step"s3_commit"{COMMIT; }
43+
step"s3_vacuum"{VACUUMFREEZEtab_freeze; }
44+
45+
# This permutation verfies that a previous bug
46+
# https://postgr.es/m/E5711E62-8FDF-4DCA-A888-C200BF6B5742@amazon.com
47+
# https://postgr.es/m/20171102112019.33wb7g5wp4zpjelu@alap3.anarazel.de
48+
# is not reintroduced. We used to make wrong pruning / freezing
49+
# decision for multixacts, which could lead to a) broken hot chains b)
50+
# dead rows being revived.
51+
permutation"s1_begin""s2_begin""s3_begin"# start transactions
52+
"s1_update""s2_key_share""s3_key_share"# have xmax be a multi with an updater, updater being oldest xid
53+
"s1_update"# create additional row version that has multis
54+
"s1_commit""s2_commit"# commit both updater and share locker
55+
"s2_vacuum"# due to bug in freezing logic, we used to *not* prune updated row, and then froze it
56+
"s1_selectone"# if hot chain is broken, the row can't be found via index scan
57+
"s3_commit"# commit remaining open xact
58+
"s2_vacuum"# pruning / freezing in broken hot chains would unset xmax, reviving rows
59+
"s1_selectall"# show borkedness

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp