Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitd372cb1

Browse files
committed
Fix handling of multixacts predating pg_upgrade
After pg_upgrade, it is possible that some tuples' Xmax have multixactscorresponding to the old installation; such multixacts cannot haverunning members anymore. In many code sites we already know not to readthem and clobber them silently, but at least when VACUUM tries to freezea multixact or determine whether one needs freezing, there's an attemptto resolve it to its member transactions by calling GetMultiXactIdMembers,and if the multixact value is "in the future" with regards to thecurrent valid multixact range, an error like this is raised: ERROR: MultiXactId 123 has not been created yet -- apparent wraparoundand vacuuming fails. Per discussion with Andrew Gierth, it is completelybogus to try to resolve multixacts coming from before a pg_upgrade,regardless of where they stand with regards to the current validmultixact range.It's possible to get from under this problem by doing SELECT FOR UPDATEof the problem tuples, but if tables are large, this is slow andtedious, so a more thorough solution is desirable.To fix, we realize that multixacts in xmax created in 9.2 and previoushave a specific bit pattern that is never used in 9.3 and later (wealready knew this, per comments and infomask tests sprinkled in variousplaces, but we weren't leveraging this knowledge appropriately).Whenever the infomask of the tuple matches that bit pattern, we justignore the multixact completely as if Xmax wasn't set; or, in the caseof tuple freezing, we act as if an unwanted value is set and clobber itwithout decoding. This guarantees that no errors will be raised, andthat the values will be progressively removed until all tables areclean. Most callers of GetMultiXactIdMembers are patched to recognizedirectly that the value is a removable "empty" multixact and avoidcalling GetMultiXactIdMembers altogether.To avoid changing the signature of GetMultiXactIdMembers() in backbranches, we keep the "allow_old" boolean flag but rename it to"from_pgupgrade"; if the flag is true, we always return an empty setinstead of looking up the multixact. (I suppose we could remove theargument in the master branch, but I chose not to do so in this commit).This was broken all along, but the error-facing message appeared firstbecause of commit 8e9a16ab8f7f and was partially fixed in a25c2b7c4db3.This fix, backpatched all the way back to 9.3, goes approximately in thesame direction as a25c2b7c4db3 but should cover all cases.Bug analysis by Andrew Gierth and Álvaro Herrera.A number of public reports match this bug:https://www.postgresql.org/message-id/20140330040029.GY4582@tamriel.snowman.nethttps://www.postgresql.org/message-id/538F3D70.6080902@publicrelay.comhttps://www.postgresql.org/message-id/556439CF.7070109@pscs.co.ukhttps://www.postgresql.org/message-id/SG2PR06MB0760098A111C88E31BD4D96FB3540@SG2PR06MB0760.apcprd06.prod.outlook.comhttps://www.postgresql.org/message-id/20160615203829.5798.4594@wrigleys.postgresql.org
1 parent07f6913 commitd372cb1

File tree

5 files changed

+88
-70
lines changed

5 files changed

+88
-70
lines changed

‎contrib/pgrowlocks/pgrowlocks.c

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -158,8 +158,7 @@ pgrowlocks(PG_FUNCTION_ARGS)
158158

159159
values[Atnum_ismulti]=pstrdup("true");
160160

161-
allow_old= !(infomask&HEAP_LOCK_MASK)&&
162-
(infomask&HEAP_XMAX_LOCK_ONLY);
161+
allow_old=HEAP_LOCKED_UPGRADED(infomask);
163162
nmembers=GetMultiXactIdMembers(xmax,&members,allow_old,
164163
false);
165164
if (nmembers==-1)

‎src/backend/access/heap/heapam.c

Lines changed: 28 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -3621,6 +3621,7 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
36213621
* HEAP_XMAX_INVALID bit set; that's fine.)
36223622
*/
36233623
if ((oldtup.t_data->t_infomask&HEAP_XMAX_INVALID)||
3624+
HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask)||
36243625
(checked_lockers&& !locker_remains))
36253626
xmax_new_tuple=InvalidTransactionId;
36263627
else
@@ -5002,8 +5003,7 @@ compute_new_xmax_infomask(TransactionId xmax, uint16 old_infomask,
50025003
* pg_upgrade; both MultiXactIdIsRunning and MultiXactIdExpand assume
50035004
* that such multis are never passed.
50045005
*/
5005-
if (!(old_infomask&HEAP_LOCK_MASK)&&
5006-
HEAP_XMAX_IS_LOCKED_ONLY(old_infomask))
5006+
if (HEAP_LOCKED_UPGRADED(old_infomask))
50075007
{
50085008
old_infomask &= ~HEAP_XMAX_IS_MULTI;
50095009
old_infomask |=HEAP_XMAX_INVALID;
@@ -5363,6 +5363,17 @@ heap_lock_updated_tuple_rec(Relation rel, ItemPointer tid, TransactionId xid,
53635363
inti;
53645364
MultiXactMember*members;
53655365

5366+
/*
5367+
* We don't need a test for pg_upgrade'd tuples: this is only
5368+
* applied to tuples after the first in an update chain. Said
5369+
* first tuple in the chain may well be locked-in-9.2-and-
5370+
* pg_upgraded, but that one was already locked by our caller,
5371+
* not us; and any subsequent ones cannot be because our
5372+
* caller must necessarily have obtained a snapshot later than
5373+
* the pg_upgrade itself.
5374+
*/
5375+
Assert(!HEAP_LOCKED_UPGRADED(mytup.t_data->t_infomask));
5376+
53665377
nmembers=GetMultiXactIdMembers(rawxmax,&members, false,
53675378
HEAP_XMAX_IS_LOCKED_ONLY(old_infomask));
53685379
for (i=0;i<nmembers;i++)
@@ -5921,14 +5932,14 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
59215932
boolhas_lockers;
59225933
TransactionIdupdate_xid;
59235934
boolupdate_committed;
5924-
boolallow_old;
59255935

59265936
*flags=0;
59275937

59285938
/* We should only be called in Multis */
59295939
Assert(t_infomask&HEAP_XMAX_IS_MULTI);
59305940

5931-
if (!MultiXactIdIsValid(multi))
5941+
if (!MultiXactIdIsValid(multi)||
5942+
HEAP_LOCKED_UPGRADED(t_infomask))
59325943
{
59335944
/* Ensure infomask bits are appropriately set/reset */
59345945
*flags |=FRM_INVALIDATE_XMAX;
@@ -5941,14 +5952,8 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
59415952
* was a locker only, it can be removed without any further
59425953
* consideration; but if it contained an update, we might need to
59435954
* preserve it.
5944-
*
5945-
* Don't assert MultiXactIdIsRunning if the multi came from a
5946-
* pg_upgrade'd share-locked tuple, though, as doing that causes an
5947-
* error to be raised unnecessarily.
59485955
*/
5949-
Assert((!(t_infomask&HEAP_LOCK_MASK)&&
5950-
HEAP_XMAX_IS_LOCKED_ONLY(t_infomask))||
5951-
!MultiXactIdIsRunning(multi,
5956+
Assert(!MultiXactIdIsRunning(multi,
59525957
HEAP_XMAX_IS_LOCKED_ONLY(t_infomask)));
59535958
if (HEAP_XMAX_IS_LOCKED_ONLY(t_infomask))
59545959
{
@@ -5990,10 +5995,8 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
59905995
* anything.
59915996
*/
59925997

5993-
allow_old= !(t_infomask&HEAP_LOCK_MASK)&&
5994-
HEAP_XMAX_IS_LOCKED_ONLY(t_infomask);
59955998
nmembers=
5996-
GetMultiXactIdMembers(multi,&members,allow_old,
5999+
GetMultiXactIdMembers(multi,&members,false,
59976000
HEAP_XMAX_IS_LOCKED_ONLY(t_infomask));
59986001
if (nmembers <=0)
59996002
{
@@ -6535,14 +6538,15 @@ static bool
65356538
DoesMultiXactIdConflict(MultiXactIdmulti,uint16infomask,
65366539
LockTupleModelockmode)
65376540
{
6538-
boolallow_old;
65396541
intnmembers;
65406542
MultiXactMember*members;
65416543
boolresult= false;
65426544
LOCKMODEwanted=tupleLockExtraInfo[lockmode].hwlock;
65436545

6544-
allow_old= !(infomask&HEAP_LOCK_MASK)&&HEAP_XMAX_IS_LOCKED_ONLY(infomask);
6545-
nmembers=GetMultiXactIdMembers(multi,&members,allow_old,
6546+
if (HEAP_LOCKED_UPGRADED(infomask))
6547+
return false;
6548+
6549+
nmembers=GetMultiXactIdMembers(multi,&members, false,
65466550
HEAP_XMAX_IS_LOCKED_ONLY(infomask));
65476551
if (nmembers >=0)
65486552
{
@@ -6625,15 +6629,15 @@ Do_MultiXactIdWait(MultiXactId multi, MultiXactStatus status,
66256629
Relationrel,ItemPointerctid,XLTW_Operoper,
66266630
int*remaining)
66276631
{
6628-
boolallow_old;
66296632
boolresult= true;
66306633
MultiXactMember*members;
66316634
intnmembers;
66326635
intremain=0;
66336636

6634-
allow_old= !(infomask&HEAP_LOCK_MASK)&&HEAP_XMAX_IS_LOCKED_ONLY(infomask);
6635-
nmembers=GetMultiXactIdMembers(multi,&members,allow_old,
6636-
HEAP_XMAX_IS_LOCKED_ONLY(infomask));
6637+
/* for pre-pg_upgrade tuples, no need to sleep at all */
6638+
nmembers=HEAP_LOCKED_UPGRADED(infomask) ?-1 :
6639+
GetMultiXactIdMembers(multi,&members, false,
6640+
HEAP_XMAX_IS_LOCKED_ONLY(infomask));
66376641

66386642
if (nmembers >=0)
66396643
{
@@ -6765,20 +6769,19 @@ heap_tuple_needs_freeze(HeapTupleHeader tuple, TransactionId cutoff_xid,
67656769
/* no xmax set, ignore */
67666770
;
67676771
}
6772+
elseif (HEAP_LOCKED_UPGRADED(tuple->t_infomask))
6773+
return true;
67686774
elseif (MultiXactIdPrecedes(multi,cutoff_multi))
67696775
return true;
67706776
else
67716777
{
67726778
MultiXactMember*members;
67736779
intnmembers;
67746780
inti;
6775-
boolallow_old;
67766781

67776782
/* need to check whether any member of the mxact is too old */
67786783

6779-
allow_old= !(tuple->t_infomask&HEAP_LOCK_MASK)&&
6780-
HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask);
6781-
nmembers=GetMultiXactIdMembers(multi,&members,allow_old,
6784+
nmembers=GetMultiXactIdMembers(multi,&members, false,
67826785
HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask));
67836786

67846787
for (i=0;i<nmembers;i++)

‎src/backend/access/transam/multixact.c

Lines changed: 26 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1181,19 +1181,25 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
11811181

11821182
/*
11831183
* GetMultiXactIdMembers
1184-
*Returns the set of MultiXactMembers that make up a MultiXactId
1185-
*
1186-
* If the given MultiXactId is older than the value we know to be oldest, we
1187-
* return -1. The caller is expected to allow that only in permissible cases,
1188-
* i.e. when the infomask lets it presuppose that the tuple had been
1189-
* share-locked before a pg_upgrade; this means that the HEAP_XMAX_LOCK_ONLY
1190-
* needs to be set, but HEAP_XMAX_KEYSHR_LOCK and HEAP_XMAX_EXCL_LOCK are not
1191-
* set.
1192-
*
1193-
* Other border conditions, such as trying to read a value that's larger than
1194-
* the value currently known as the next to assign, raise an error. Previously
1195-
* these also returned -1, but since this can lead to the wrong visibility
1196-
* results, it is dangerous to do that.
1184+
*Return the set of MultiXactMembers that make up a MultiXactId
1185+
*
1186+
* Return value is the number of members found, or -1 if there are none,
1187+
* and *members is set to a newly palloc'ed array of members. It's the
1188+
* caller's responsibility to free it when done with it.
1189+
*
1190+
* from_pgupgrade must be passed as true if and only if only the multixact
1191+
* corresponds to a value from a tuple that was locked in a 9.2-or-older
1192+
* installation and later pg_upgrade'd (that is, the infomask is
1193+
* HEAP_LOCKED_UPGRADED). In this case, we know for certain that no members
1194+
* can still be running, so we return -1 just like for an empty multixact
1195+
* without any further checking. It would be wrong to try to resolve such a
1196+
* multixact: either the multixact is within the current valid multixact
1197+
* range, in which case the returned result would be bogus, or outside that
1198+
* range, in which case an error would be raised.
1199+
*
1200+
* In all other cases, the passed multixact must be within the known valid
1201+
* range, that is, greater to or equal than oldestMultiXactId, and less than
1202+
* nextMXact. Otherwise, an error is raised.
11971203
*
11981204
* onlyLock must be set to true if caller is certain that the given multi
11991205
* is used only to lock tuples; can be false without loss of correctness,
@@ -1202,7 +1208,7 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
12021208
*/
12031209
int
12041210
GetMultiXactIdMembers(MultiXactIdmulti,MultiXactMember**members,
1205-
boolallow_old,boolonlyLock)
1211+
boolfrom_pgupgrade,boolonlyLock)
12061212
{
12071213
intpageno;
12081214
intprev_pageno;
@@ -1221,7 +1227,7 @@ GetMultiXactIdMembers(MultiXactId multi, MultiXactMember **members,
12211227

12221228
debug_elog3(DEBUG2,"GetMembers: asked for %u",multi);
12231229

1224-
if (!MultiXactIdIsValid(multi))
1230+
if (!MultiXactIdIsValid(multi)||from_pgupgrade)
12251231
return-1;
12261232

12271233
/* See if the MultiXactId is in the local cache */
@@ -1254,18 +1260,11 @@ GetMultiXactIdMembers(MultiXactId multi, MultiXactMember **members,
12541260
*
12551261
* An ID older than MultiXactState->oldestMultiXactId cannot possibly be
12561262
* useful; it has already been removed, or will be removed shortly, by
1257-
* truncation. Returning the wrong values could lead to an incorrect
1258-
* visibility result. However, to support pg_upgrade we need to allow an
1259-
* empty set to be returned regardless, if the caller is willing to accept
1260-
* it; the caller is expected to check that it's an allowed condition
1261-
* (such as ensuring that the infomask bits set on the tuple are
1262-
* consistent with the pg_upgrade scenario). If the caller is expecting
1263-
* this to be called only on recently created multis, then we raise an
1264-
* error.
1263+
* truncation. If one is passed, an error is raised.
12651264
*
1266-
*Conversely, an ID >= nextMXact shouldn't ever be seen here; if it is
1267-
*seen, itimplies undetected ID wraparound has occurred. This raises a
1268-
*harderror.
1265+
*Also, an ID >= nextMXact shouldn't ever be seen here; if it is seen, it
1266+
* implies undetected ID wraparound has occurred. This raises a hard
1267+
* error.
12691268
*
12701269
* Shared lock is enough here since we aren't modifying any global state.
12711270
* Acquire it just long enough to grab the current counter values. We may
@@ -1281,7 +1280,7 @@ GetMultiXactIdMembers(MultiXactId multi, MultiXactMember **members,
12811280

12821281
if (MultiXactIdPrecedes(multi,oldestMXact))
12831282
{
1284-
ereport(allow_old ?DEBUG1 :ERROR,
1283+
ereport(ERROR,
12851284
(errcode(ERRCODE_INTERNAL_ERROR),
12861285
errmsg("MultiXactId %u does no longer exist -- apparent wraparound",
12871286
multi)));

‎src/backend/utils/time/tqual.c

Lines changed: 8 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -609,15 +609,12 @@ HeapTupleSatisfiesUpdate(HeapTuple htup, CommandId curcid,
609609
{
610610
TransactionIdxmax;
611611

612+
if (HEAP_LOCKED_UPGRADED(tuple->t_infomask))
613+
returnHeapTupleMayBeUpdated;
614+
612615
if (HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask))
613616
{
614-
/*
615-
* If it's only locked but neither EXCL_LOCK nor KEYSHR_LOCK is
616-
* set, it cannot possibly be running. Otherwise need to check.
617-
*/
618-
if ((tuple->t_infomask& (HEAP_XMAX_EXCL_LOCK |
619-
HEAP_XMAX_KEYSHR_LOCK))&&
620-
MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple), true))
617+
if (MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple), true))
621618
returnHeapTupleBeingUpdated;
622619

623620
SetHintBits(tuple,buffer,HEAP_XMAX_INVALID,InvalidTransactionId);
@@ -1258,26 +1255,21 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin,
12581255
* "Deleting" xact really only locked it, so the tuple is live in any
12591256
* case. However, we should make sure that either XMAX_COMMITTED or
12601257
* XMAX_INVALID gets set once the xact is gone, to reduce the costs of
1261-
* examining the tuple for future xacts. Also, marking dead
1262-
* MultiXacts as invalid here provides defense against MultiXactId
1263-
* wraparound (see also comments in heap_freeze_tuple()).
1258+
* examining the tuple for future xacts.
12641259
*/
12651260
if (!(tuple->t_infomask&HEAP_XMAX_COMMITTED))
12661261
{
12671262
if (tuple->t_infomask&HEAP_XMAX_IS_MULTI)
12681263
{
12691264
/*
1270-
* If it's only locked but neither EXCL_LOCK nor KEYSHR_LOCK
1271-
* are set, it cannot possibly be running; otherwise have to
1272-
* check.
1265+
* If it's a pre-pg_upgrade tuple, the multixact cannot
1266+
* possibly be running; otherwise have to check.
12731267
*/
1274-
if ((tuple->t_infomask& (HEAP_XMAX_EXCL_LOCK |
1275-
HEAP_XMAX_KEYSHR_LOCK))&&
1268+
if (!HEAP_LOCKED_UPGRADED(tuple->t_infomask)&&
12761269
MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple),
12771270
true))
12781271
returnHEAPTUPLE_LIVE;
12791272
SetHintBits(tuple,buffer,HEAP_XMAX_INVALID,InvalidTransactionId);
1280-
12811273
}
12821274
else
12831275
{

‎src/include/access/htup_details.h

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,31 @@ struct HeapTupleHeaderData
217217
(((infomask) & HEAP_XMAX_LOCK_ONLY) || \
218218
(((infomask) & (HEAP_XMAX_IS_MULTI | HEAP_LOCK_MASK)) == HEAP_XMAX_EXCL_LOCK))
219219

220+
/*
221+
* A tuple that has HEAP_XMAX_IS_MULTI and HEAP_XMAX_LOCK_ONLY but neither of
222+
* XMAX_EXCL_LOCK and XMAX_KEYSHR_LOCK must come from a tuple that was
223+
* share-locked in 9.2 or earlier and then pg_upgrade'd.
224+
*
225+
* In 9.2 and prior, HEAP_XMAX_IS_MULTI was only set when there were multiple
226+
* FOR SHARE lockers of that tuple. That set HEAP_XMAX_LOCK_ONLY (with a
227+
* different name back then) but neither of HEAP_XMAX_EXCL_LOCK and
228+
* HEAP_XMAX_KEYSHR_LOCK. That combination is no longer possible in 9.3 and
229+
* up, so if we see that combination we know for certain that the tuple was
230+
* locked in an earlier release; since all such lockers are gone (they cannot
231+
* survive through pg_upgrade), such tuples can safely be considered not
232+
* locked.
233+
*
234+
* We must not resolve such multixacts locally, because the result would be
235+
* bogus, regardless of where they stand with respect to the current valid
236+
* multixact range.
237+
*/
238+
#defineHEAP_LOCKED_UPGRADED(infomask) \
239+
( \
240+
((infomask) & HEAP_XMAX_IS_MULTI) && \
241+
((infomask) & HEAP_XMAX_LOCK_ONLY) && \
242+
(((infomask) & (HEAP_XMAX_EXCL_LOCK | HEAP_XMAX_KEYSHR_LOCK)) == 0) \
243+
)
244+
220245
/*
221246
* Use these to test whether a particular lock is applied to a tuple
222247
*/

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp