Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit78db307

Browse files
committed
Defend against bad relfrozenxid/relminmxid/datfrozenxid/datminmxid values.
In commita61daa1, we fixed pg_upgrade sothat it would install sane relminmxid and datminmxid values, but that doesnot cure the problem for installations that were already pg_upgraded to9.3; they'll initially have "1" in those fields. This is not a big problemso long as 1 is "in the past" compared to the current nextMultiXactcounter. But if an installation were more than halfway to the MXID wrappoint at the time of upgrade, 1 would appear to be "in the future" andthat would effectively disable tracking of oldest MXIDs in thosetables/databases, until such time as the counter wrapped around.While in itself this isn't worse than the situation pre-9.3, where we didnot manage MXID wraparound risk at all, the consequences of prematuretruncation of pg_multixact are worse now; so we ought to make some effortto cope with this. We discussed advising users to fix the tracking valuesmanually, but that seems both very tedious and very error-prone.Instead, this patch adopts two amelioration rules. First, a relminmxidvalue that is "in the future" is allowed to be overwritten with afull-table VACUUM's actual freeze cutoff, ignoring the normal rule thatrelminmxid should never go backwards. (This essentially assumes that wehave enough defenses in place that wraparound can never occur anymore,and thus that a value "in the future" must be corrupt.) Second, if we seeany "in the future" values then we refrain from truncating pg_clog andpg_multixact. This prevents loss of clog data until we have cleaned upall the broken tracking data. In the worst case that could result inconsiderable clog bloat, but in practice we expect that relfrozenxid-drivenfreezing will happen soon enough to fix the problem before clog bloatbecomes intolerable. (Users could do manual VACUUM FREEZEs if not.)Note that this mechanism cannot save us if there are already-wrapped oralready-truncated-away MXIDs in the table; it's only capable of dealingwith corrupt tracking values. But that's the situation we have with thepg_upgrade bug.For consistency, apply the same rules to relfrozenxid/datfrozenxid. Thereare not known mechanisms for these to get messed up, but if they were, thesame tactics seem appropriate for fixing them.
1 parent4c3c911 commit78db307

File tree

1 file changed

+99
-26
lines changed

1 file changed

+99
-26
lines changed

‎src/backend/commands/vacuum.c

Lines changed: 99 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,10 @@ static BufferAccessStrategy vac_strategy;
6666

6767
/* non-export function prototypes */
6868
staticList*get_rel_oids(Oidrelid,constRangeVar*vacrel);
69-
staticvoidvac_truncate_clog(TransactionIdfrozenXID,MultiXactIdminMulti);
69+
staticvoidvac_truncate_clog(TransactionIdfrozenXID,
70+
MultiXactIdminMulti,
71+
TransactionIdlastSaneFrozenXid,
72+
MultiXactIdlastSaneMinMulti);
7073
staticboolvacuum_rel(Oidrelid,VacuumStmt*vacstmt,booldo_toast,
7174
boolfor_wraparound);
7275

@@ -733,19 +736,33 @@ vac_update_relstats(Relation relation,
733736
}
734737

735738
/*
736-
* relfrozenxid should never go backward. Caller can pass
737-
* InvalidTransactionId if it has no new data.
739+
* Update relfrozenxid, unless caller passed InvalidTransactionId
740+
* indicating it has no new data.
741+
*
742+
* Ordinarily, we don't let relfrozenxid go backwards: if things are
743+
* working correctly, the only way the new frozenxid could be older would
744+
* be if a previous VACUUM was done with a tighter freeze_min_age, in
745+
* which case we don't want to forget the work it already did. However,
746+
* if the stored relfrozenxid is "in the future", then it must be corrupt
747+
* and it seems best to overwrite it with the cutoff we used this time.
748+
* See vac_update_datfrozenxid() concerning what we consider to be "in the
749+
* future".
738750
*/
739751
if (TransactionIdIsNormal(frozenxid)&&
740-
TransactionIdPrecedes(pgcform->relfrozenxid,frozenxid))
752+
pgcform->relfrozenxid!=frozenxid&&
753+
(TransactionIdPrecedes(pgcform->relfrozenxid,frozenxid)||
754+
TransactionIdPrecedes(GetOldestXmin(NULL, true),
755+
pgcform->relfrozenxid)))
741756
{
742757
pgcform->relfrozenxid=frozenxid;
743758
dirty= true;
744759
}
745760

746-
/*relminmxid must never go backward, either */
761+
/*Similarly for relminmxid */
747762
if (MultiXactIdIsValid(minmulti)&&
748-
MultiXactIdPrecedes(pgcform->relminmxid,minmulti))
763+
pgcform->relminmxid!=minmulti&&
764+
(MultiXactIdPrecedes(pgcform->relminmxid,minmulti)||
765+
MultiXactIdPrecedes(GetOldestMultiXactId(),pgcform->relminmxid)))
749766
{
750767
pgcform->relminmxid=minmulti;
751768
dirty= true;
@@ -772,8 +789,8 @@ vac_update_relstats(Relation relation,
772789
*truncate pg_clog and pg_multixact.
773790
*
774791
*We violate transaction semantics here by overwriting the database's
775-
*existing pg_database tuple with the newvalue. This is reasonably
776-
*safe since the newvalue is correct whether or not this transaction
792+
*existing pg_database tuple with the newvalues. This is reasonably
793+
*safe since the newvalues are correct whether or not this transaction
777794
*commits. As with vac_update_relstats, this avoids leaving dead tuples
778795
*behind after a VACUUM.
779796
*/
@@ -786,7 +803,10 @@ vac_update_datfrozenxid(void)
786803
SysScanDescscan;
787804
HeapTupleclassTup;
788805
TransactionIdnewFrozenXid;
806+
TransactionIdlastSaneFrozenXid;
789807
MultiXactIdnewMinMulti;
808+
MultiXactIdlastSaneMinMulti;
809+
boolbogus= false;
790810
booldirty= false;
791811

792812
/*
@@ -795,13 +815,13 @@ vac_update_datfrozenxid(void)
795815
* committed pg_class entries for new tables; see AddNewRelationTuple().
796816
* So we cannot produce a wrong minimum by starting with this.
797817
*/
798-
newFrozenXid=GetOldestXmin(NULL, true);
818+
newFrozenXid=lastSaneFrozenXid=GetOldestXmin(NULL, true);
799819

800820
/*
801821
* Similarly, initialize the MultiXact "min" with the value that would be
802822
* used on pg_class for new tables. See AddNewRelationTuple().
803823
*/
804-
newMinMulti=GetOldestMultiXactId();
824+
newMinMulti=lastSaneMinMulti=GetOldestMultiXactId();
805825

806826
/*
807827
* We must seqscan pg_class to find the minimum Xid, because there is no
@@ -828,6 +848,21 @@ vac_update_datfrozenxid(void)
828848
Assert(TransactionIdIsNormal(classForm->relfrozenxid));
829849
Assert(MultiXactIdIsValid(classForm->relminmxid));
830850

851+
/*
852+
* If things are working properly, no relation should have a
853+
* relfrozenxid or relminmxid that is "in the future". However, such
854+
* cases have been known to arise due to bugs in pg_upgrade. If we
855+
* see any entries that are "in the future", chicken out and don't do
856+
* anything. This ensures we won't truncate clog before those
857+
* relations have been scanned and cleaned up.
858+
*/
859+
if (TransactionIdPrecedes(lastSaneFrozenXid,classForm->relfrozenxid)||
860+
MultiXactIdPrecedes(lastSaneMinMulti,classForm->relminmxid))
861+
{
862+
bogus= true;
863+
break;
864+
}
865+
831866
if (TransactionIdPrecedes(classForm->relfrozenxid,newFrozenXid))
832867
newFrozenXid=classForm->relfrozenxid;
833868

@@ -839,6 +874,10 @@ vac_update_datfrozenxid(void)
839874
systable_endscan(scan);
840875
heap_close(relation,AccessShareLock);
841876

877+
/* chicken out if bogus data found */
878+
if (bogus)
879+
return;
880+
842881
Assert(TransactionIdIsNormal(newFrozenXid));
843882
Assert(MultiXactIdIsValid(newMinMulti));
844883

@@ -852,21 +891,30 @@ vac_update_datfrozenxid(void)
852891
dbform= (Form_pg_database)GETSTRUCT(tuple);
853892

854893
/*
855-
* Don't allow datfrozenxid to go backward (probably can't happen anyway);
856-
* and detect the common case where it doesn't go forward either.
894+
* As in vac_update_relstats(), we ordinarily don't want to let
895+
* datfrozenxid go backward; but if it's "in the future" then it must be
896+
* corrupt and it seems best to overwrite it.
857897
*/
858-
if (TransactionIdPrecedes(dbform->datfrozenxid,newFrozenXid))
898+
if (dbform->datfrozenxid!=newFrozenXid&&
899+
(TransactionIdPrecedes(dbform->datfrozenxid,newFrozenXid)||
900+
TransactionIdPrecedes(lastSaneFrozenXid,dbform->datfrozenxid)))
859901
{
860902
dbform->datfrozenxid=newFrozenXid;
861903
dirty= true;
862904
}
905+
else
906+
newFrozenXid=dbform->datfrozenxid;
863907

864-
/* ditto */
865-
if (MultiXactIdPrecedes(dbform->datminmxid,newMinMulti))
908+
/* Ditto for datminmxid */
909+
if (dbform->datminmxid!=newMinMulti&&
910+
(MultiXactIdPrecedes(dbform->datminmxid,newMinMulti)||
911+
MultiXactIdPrecedes(lastSaneMinMulti,dbform->datminmxid)))
866912
{
867913
dbform->datminmxid=newMinMulti;
868914
dirty= true;
869915
}
916+
else
917+
newMinMulti=dbform->datminmxid;
870918

871919
if (dirty)
872920
heap_inplace_update(relation,tuple);
@@ -875,12 +923,13 @@ vac_update_datfrozenxid(void)
875923
heap_close(relation,RowExclusiveLock);
876924

877925
/*
878-
* If we were able to advance datfrozenxid, see if we can truncate
879-
* pg_clog.Also do it if the shared XID-wrap-limit info is stale, since
880-
* this action will update that too.
926+
* If we were able to advance datfrozenxid or datminmxid, see if we can
927+
*truncatepg_clog and/or pg_multixact.Also do it if the shared
928+
*XID-wrap-limit info is stale, sincethis action will update that too.
881929
*/
882930
if (dirty||ForceTransactionIdLimitUpdate())
883-
vac_truncate_clog(newFrozenXid,newMinMulti);
931+
vac_truncate_clog(newFrozenXid,newMinMulti,
932+
lastSaneFrozenXid,lastSaneMinMulti);
884933
}
885934

886935

@@ -890,31 +939,38 @@ vac_update_datfrozenxid(void)
890939
*Scan pg_database to determine the system-wide oldest datfrozenxid,
891940
*and use it to truncate the transaction commit log (pg_clog).
892941
*Also update the XID wrap limit info maintained by varsup.c.
942+
*Likewise for datminmxid.
893943
*
894-
*The passed XID is simply the one I just wrote into my pg_database
895-
*entry. It's used to initialize the "min" calculation.
944+
*The passed frozenXID and minMulti are the updated values for my own
945+
*pg_database entry. They're used to initialize the "min" calculations.
946+
*The caller also passes the "last sane" XID and MXID, since it has
947+
*those at hand already.
896948
*
897949
*This routine is only invoked when we've managed to change our
898-
*DB's datfrozenxid entry, or we found that the shared XID-wrap-limit
899-
*info is stale.
950+
*DB's datfrozenxid/datminmxid values, or we found that the shared
951+
*XID-wrap-limitinfo is stale.
900952
*/
901953
staticvoid
902-
vac_truncate_clog(TransactionIdfrozenXID,MultiXactIdminMulti)
954+
vac_truncate_clog(TransactionIdfrozenXID,
955+
MultiXactIdminMulti,
956+
TransactionIdlastSaneFrozenXid,
957+
MultiXactIdlastSaneMinMulti)
903958
{
904959
TransactionIdmyXID=GetCurrentTransactionId();
905960
Relationrelation;
906961
HeapScanDescscan;
907962
HeapTupletuple;
908963
Oidoldestxid_datoid;
909964
Oidminmulti_datoid;
965+
boolbogus= false;
910966
boolfrozenAlreadyWrapped= false;
911967

912-
/* init oldest datoids to sync with myfrozen values */
968+
/* init oldest datoids to sync with myfrozenXID/minMulti values */
913969
oldestxid_datoid=MyDatabaseId;
914970
minmulti_datoid=MyDatabaseId;
915971

916972
/*
917-
* Scan pg_database to compute the minimum datfrozenxid
973+
* Scan pg_database to compute the minimum datfrozenxid/datminmxid
918974
*
919975
* Note: we need not worry about a race condition with new entries being
920976
* inserted by CREATE DATABASE. Any such entry will have a copy of some
@@ -936,6 +992,19 @@ vac_truncate_clog(TransactionId frozenXID, MultiXactId minMulti)
936992
Assert(TransactionIdIsNormal(dbform->datfrozenxid));
937993
Assert(MultiXactIdIsValid(dbform->datminmxid));
938994

995+
/*
996+
* If things are working properly, no database should have a
997+
* datfrozenxid or datminmxid that is "in the future". However, such
998+
* cases have been known to arise due to bugs in pg_upgrade. If we
999+
* see any entries that are "in the future", chicken out and don't do
1000+
* anything. This ensures we won't truncate clog before those
1001+
* databases have been scanned and cleaned up. (We will issue the
1002+
* "already wrapped" warning if appropriate, though.)
1003+
*/
1004+
if (TransactionIdPrecedes(lastSaneFrozenXid,dbform->datfrozenxid)||
1005+
MultiXactIdPrecedes(lastSaneMinMulti,dbform->datminmxid))
1006+
bogus= true;
1007+
9391008
if (TransactionIdPrecedes(myXID,dbform->datfrozenxid))
9401009
frozenAlreadyWrapped= true;
9411010
elseif (TransactionIdPrecedes(dbform->datfrozenxid,frozenXID))
@@ -969,6 +1038,10 @@ vac_truncate_clog(TransactionId frozenXID, MultiXactId minMulti)
9691038
return;
9701039
}
9711040

1041+
/* chicken out if data is bogus in any other way */
1042+
if (bogus)
1043+
return;
1044+
9721045
/*
9731046
* Truncate CLOG to the oldest computed value. Note we don't truncate
9741047
* multixacts; that will be done by the next checkpoint.

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp