Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit8523492

Browse files
Remove tupgone special case from vacuumlazy.c.
Retry the call to heap_prune_page() in rare cases where there isdisagreement between the heap_prune_page() call and the call toHeapTupleSatisfiesVacuum() that immediately follows. Disagreement ispossible when a concurrently-aborted transaction makes a tuple DEADduring the tiny window between each step. This was the only case wherea tuple considered DEAD by VACUUM still had storage following pruning.VACUUM's definition of dead tuples is now uniformly simple andunambiguous: dead tuples from each page are always LP_DEAD line pointersthat were encountered just after we performed pruning (and just beforewe considered freezing remaining items with tuple storage).Eliminating the tupgone=true special case enables INDEX_CLEANUP=offstyle skipping of index vacuuming that takes place based on flexible,dynamic criteria. The INDEX_CLEANUP=off case had to know about skippingindexes up-front before now, due to a subtle interaction with thespecial case (see commitdd69597) -- this was a special case untoitself. Now there are no special cases. And so now it won't matterwhen or how we decide to skip index vacuuming: it won't affect howpruning behaves, and it won't be affected by any of the implementationdetails of pruning or freezing.Also remove XLOG_HEAP2_CLEANUP_INFO records. These are no longernecessary because we now rely entirely on heap pruning taking care ofrecovery conflicts. There is no longer any need to generate recoveryconflicts for DEAD tuples that pruning just missed. This also meansthat heap vacuuming now uses exactly the same strategy for recoveryconflicts as index vacuuming always has: REDO routines never need toprocess a latestRemovedXid from the WAL record, since earlier REDO ofthe WAL record from pruning is sufficient in all cases. The genericXLOG_HEAP2_CLEAN record type is now split into two new record types toreflect this new division (these are called XLOG_HEAP2_PRUNE andXLOG_HEAP2_VACUUM).Also stop acquiring a super-exclusive lock for heap pages when they'revacuumed during VACUUM's second heap pass. A regular exclusive lock isenough. This is correct because heap page vacuuming is now strictly amatter of setting the LP_DEAD line pointers to LP_UNUSED. No otherbackend can have a pointer to a tuple located in a pinned buffer thatcan be invalidated by a concurrent heap page vacuum operation.Heap vacuuming can now be thought of as conceptually similar to indexvacuuming and conceptually dissimilar to heap pruning. Heap pruning nowhas sole responsibility for anything involving the logical contents ofthe database (e.g., managing transaction status information, recoveryconflicts, considering what to do with HOT chains). Index vacuuming andheap vacuuming are now only concerned with recycling garbage items fromphysical data structures that back the logical database.Bump XLOG_PAGE_MAGIC due to pruning and heap page vacuum WAL recordchanges.Credit for the idea of retrying pruning a page to avoid the tupgone casegoes to Andres Freund.Author: Peter Geoghegan <pg@bowt.ie>Reviewed-By: Andres Freund <andres@anarazel.de>Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>Discussion:https://postgr.es/m/CAH2-WznneCXTzuFmcwx_EyRQgfsfJAAsu+CsqRFmFXCAar=nJw@mail.gmail.com
1 parent789d81d commit8523492

File tree

12 files changed

+285
-336
lines changed

12 files changed

+285
-336
lines changed

‎src/backend/access/gist/gistxlog.c

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -184,10 +184,10 @@ gistRedoDeleteRecord(XLogReaderState *record)
184184
*
185185
* GiST delete records can conflict with standby queries. You might think
186186
* that vacuum records would conflict as well, but we've handled that
187-
* already.XLOG_HEAP2_CLEANUP_INFO records provide the highest xid
188-
*cleaned bythe vacuum of the heap and so we can resolve any conflicts
189-
*just oncewhen that arrives. After that we know that no conflicts
190-
*exist fromindividual gist vacuum records on that index.
187+
* already.XLOG_HEAP2_PRUNE records provide the highest xid cleaned by
188+
* the vacuum of the heap and so we can resolve any conflicts just once
189+
* when that arrives. After that we know that no conflicts exist from
190+
* individual gist vacuum records on that index.
191191
*/
192192
if (InHotStandby)
193193
{

‎src/backend/access/hash/hash_xlog.c

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -992,10 +992,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record)
992992
* Hash index records that are marked as LP_DEAD and being removed during
993993
* hash index tuple insertion can conflict with standby queries. You might
994994
* think that vacuum records would conflict as well, but we've handled
995-
* that already.XLOG_HEAP2_CLEANUP_INFO records provide the highest xid
996-
*cleanedby the vacuum of the heap and so we can resolve any conflicts
997-
*just oncewhen that arrives. After that we know that no conflicts
998-
*exist fromindividual hash index vacuum records on that index.
995+
* that already.XLOG_HEAP2_PRUNE records provide the highest xid cleaned
996+
* by the vacuum of the heap and so we can resolve any conflicts just once
997+
* when that arrives. After that we know that no conflicts exist from
998+
* individual hash index vacuum records on that index.
999999
*/
10001000
if (InHotStandby)
10011001
{

‎src/backend/access/heap/heapam.c

Lines changed: 87 additions & 118 deletions
Original file line numberDiff line numberDiff line change
@@ -7538,7 +7538,7 @@ heap_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate)
75387538
* must have considered the original tuple header as part of
75397539
* generating its own latestRemovedXid value.
75407540
*
7541-
* Relying onXLOG_HEAP2_CLEAN records like this is the same
7541+
* Relying onXLOG_HEAP2_PRUNE records like this is the same
75427542
* strategy that index vacuuming uses in all cases. Index VACUUM
75437543
* WAL records don't even have a latestRemovedXid field of their
75447544
* own for this reason.
@@ -7957,88 +7957,6 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate)
79577957
returnnblocksfavorable;
79587958
}
79597959

7960-
/*
7961-
* Perform XLogInsert to register a heap cleanup info message. These
7962-
* messages are sent once per VACUUM and are required because
7963-
* of the phasing of removal operations during a lazy VACUUM.
7964-
* see comments for vacuum_log_cleanup_info().
7965-
*/
7966-
XLogRecPtr
7967-
log_heap_cleanup_info(RelFileNodernode,TransactionIdlatestRemovedXid)
7968-
{
7969-
xl_heap_cleanup_infoxlrec;
7970-
XLogRecPtrrecptr;
7971-
7972-
xlrec.node=rnode;
7973-
xlrec.latestRemovedXid=latestRemovedXid;
7974-
7975-
XLogBeginInsert();
7976-
XLogRegisterData((char*)&xlrec,SizeOfHeapCleanupInfo);
7977-
7978-
recptr=XLogInsert(RM_HEAP2_ID,XLOG_HEAP2_CLEANUP_INFO);
7979-
7980-
returnrecptr;
7981-
}
7982-
7983-
/*
7984-
* Perform XLogInsert for a heap-clean operation. Caller must already
7985-
* have modified the buffer and marked it dirty.
7986-
*
7987-
* Note: prior to Postgres 8.3, the entries in the nowunused[] array were
7988-
* zero-based tuple indexes. Now they are one-based like other uses
7989-
* of OffsetNumber.
7990-
*
7991-
* We also include latestRemovedXid, which is the greatest XID present in
7992-
* the removed tuples. That allows recovery processing to cancel or wait
7993-
* for long standby queries that can still see these tuples.
7994-
*/
7995-
XLogRecPtr
7996-
log_heap_clean(Relationreln,Bufferbuffer,
7997-
OffsetNumber*redirected,intnredirected,
7998-
OffsetNumber*nowdead,intndead,
7999-
OffsetNumber*nowunused,intnunused,
8000-
TransactionIdlatestRemovedXid)
8001-
{
8002-
xl_heap_cleanxlrec;
8003-
XLogRecPtrrecptr;
8004-
8005-
/* Caller should not call me on a non-WAL-logged relation */
8006-
Assert(RelationNeedsWAL(reln));
8007-
8008-
xlrec.latestRemovedXid=latestRemovedXid;
8009-
xlrec.nredirected=nredirected;
8010-
xlrec.ndead=ndead;
8011-
8012-
XLogBeginInsert();
8013-
XLogRegisterData((char*)&xlrec,SizeOfHeapClean);
8014-
8015-
XLogRegisterBuffer(0,buffer,REGBUF_STANDARD);
8016-
8017-
/*
8018-
* The OffsetNumber arrays are not actually in the buffer, but we pretend
8019-
* that they are. When XLogInsert stores the whole buffer, the offset
8020-
* arrays need not be stored too. Note that even if all three arrays are
8021-
* empty, we want to expose the buffer as a candidate for whole-page
8022-
* storage, since this record type implies a defragmentation operation
8023-
* even if no line pointers changed state.
8024-
*/
8025-
if (nredirected>0)
8026-
XLogRegisterBufData(0, (char*)redirected,
8027-
nredirected*sizeof(OffsetNumber)*2);
8028-
8029-
if (ndead>0)
8030-
XLogRegisterBufData(0, (char*)nowdead,
8031-
ndead*sizeof(OffsetNumber));
8032-
8033-
if (nunused>0)
8034-
XLogRegisterBufData(0, (char*)nowunused,
8035-
nunused*sizeof(OffsetNumber));
8036-
8037-
recptr=XLogInsert(RM_HEAP2_ID,XLOG_HEAP2_CLEAN);
8038-
8039-
returnrecptr;
8040-
}
8041-
80427960
/*
80437961
* Perform XLogInsert for a heap-freeze operation. Caller must have already
80447962
* modified the buffer and marked it dirty.
@@ -8510,34 +8428,15 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_changed,
85108428
}
85118429

85128430
/*
8513-
* Handles CLEANUP_INFO
8514-
*/
8515-
staticvoid
8516-
heap_xlog_cleanup_info(XLogReaderState*record)
8517-
{
8518-
xl_heap_cleanup_info*xlrec= (xl_heap_cleanup_info*)XLogRecGetData(record);
8519-
8520-
if (InHotStandby)
8521-
ResolveRecoveryConflictWithSnapshot(xlrec->latestRemovedXid,xlrec->node);
8522-
8523-
/*
8524-
* Actual operation is a no-op. Record type exists to provide a means for
8525-
* conflict processing to occur before we begin index vacuum actions. see
8526-
* vacuumlazy.c and also comments in btvacuumpage()
8527-
*/
8528-
8529-
/* Backup blocks are not used in cleanup_info records */
8530-
Assert(!XLogRecHasAnyBlockRefs(record));
8531-
}
8532-
8533-
/*
8534-
* Handles XLOG_HEAP2_CLEAN record type
8431+
* Handles XLOG_HEAP2_PRUNE record type.
8432+
*
8433+
* Acquires a super-exclusive lock.
85358434
*/
85368435
staticvoid
8537-
heap_xlog_clean(XLogReaderState*record)
8436+
heap_xlog_prune(XLogReaderState*record)
85388437
{
85398438
XLogRecPtrlsn=record->EndRecPtr;
8540-
xl_heap_clean*xlrec= (xl_heap_clean*)XLogRecGetData(record);
8439+
xl_heap_prune*xlrec= (xl_heap_prune*)XLogRecGetData(record);
85418440
Bufferbuffer;
85428441
RelFileNodernode;
85438442
BlockNumberblkno;
@@ -8548,12 +8447,8 @@ heap_xlog_clean(XLogReaderState *record)
85488447
/*
85498448
* We're about to remove tuples. In Hot Standby mode, ensure that there's
85508449
* no queries running for which the removed tuples are still visible.
8551-
*
8552-
* Not all HEAP2_CLEAN records remove tuples with xids, so we only want to
8553-
* conflict on the records that cause MVCC failures for user queries. If
8554-
* latestRemovedXid is invalid, skip conflict processing.
85558450
*/
8556-
if (InHotStandby&&TransactionIdIsValid(xlrec->latestRemovedXid))
8451+
if (InHotStandby)
85578452
ResolveRecoveryConflictWithSnapshot(xlrec->latestRemovedXid,rnode);
85588453

85598454
/*
@@ -8606,7 +8501,7 @@ heap_xlog_clean(XLogReaderState *record)
86068501
UnlockReleaseBuffer(buffer);
86078502

86088503
/*
8609-
* Aftercleaning records from a page, it's useful to update the FSM
8504+
* Afterpruning records from a page, it's useful to update the FSM
86108505
* about it, as it may cause the page become target for insertions
86118506
* later even if vacuum decides not to visit it (which is possible if
86128507
* gets marked all-visible.)
@@ -8618,6 +8513,80 @@ heap_xlog_clean(XLogReaderState *record)
86188513
}
86198514
}
86208515

8516+
/*
8517+
* Handles XLOG_HEAP2_VACUUM record type.
8518+
*
8519+
* Acquires an exclusive lock only.
8520+
*/
8521+
staticvoid
8522+
heap_xlog_vacuum(XLogReaderState*record)
8523+
{
8524+
XLogRecPtrlsn=record->EndRecPtr;
8525+
xl_heap_vacuum*xlrec= (xl_heap_vacuum*)XLogRecGetData(record);
8526+
Bufferbuffer;
8527+
BlockNumberblkno;
8528+
XLogRedoActionaction;
8529+
8530+
/*
8531+
* If we have a full-page image, restore it(without using a cleanup lock)
8532+
* and we're done.
8533+
*/
8534+
action=XLogReadBufferForRedoExtended(record,0,RBM_NORMAL, false,
8535+
&buffer);
8536+
if (action==BLK_NEEDS_REDO)
8537+
{
8538+
Pagepage= (Page)BufferGetPage(buffer);
8539+
OffsetNumber*nowunused;
8540+
Sizedatalen;
8541+
OffsetNumber*offnum;
8542+
8543+
nowunused= (OffsetNumber*)XLogRecGetBlockData(record,0,&datalen);
8544+
8545+
/* Shouldn't be a record unless there's something to do */
8546+
Assert(xlrec->nunused>0);
8547+
8548+
/* Update all now-unused line pointers */
8549+
offnum=nowunused;
8550+
for (inti=0;i<xlrec->nunused;i++)
8551+
{
8552+
OffsetNumberoff=*offnum++;
8553+
ItemIdlp=PageGetItemId(page,off);
8554+
8555+
Assert(ItemIdIsDead(lp)&& !ItemIdHasStorage(lp));
8556+
ItemIdSetUnused(lp);
8557+
}
8558+
8559+
/*
8560+
* Update the page's hint bit about whether it has free pointers
8561+
*/
8562+
PageSetHasFreeLinePointers(page);
8563+
8564+
PageSetLSN(page,lsn);
8565+
MarkBufferDirty(buffer);
8566+
}
8567+
8568+
if (BufferIsValid(buffer))
8569+
{
8570+
Sizefreespace=PageGetHeapFreeSpace(BufferGetPage(buffer));
8571+
RelFileNodernode;
8572+
8573+
XLogRecGetBlockTag(record,0,&rnode,NULL,&blkno);
8574+
8575+
UnlockReleaseBuffer(buffer);
8576+
8577+
/*
8578+
* After vacuuming LP_DEAD items from a page, it's useful to update
8579+
* the FSM about it, as it may cause the page become target for
8580+
* insertions later even if vacuum decides not to visit it (which is
8581+
* possible if gets marked all-visible.)
8582+
*
8583+
* Do this regardless of a full-page image being applied, since the
8584+
* FSM data is not in the page anyway.
8585+
*/
8586+
XLogRecordPageWithFreeSpace(rnode,blkno,freespace);
8587+
}
8588+
}
8589+
86218590
/*
86228591
* Replay XLOG_HEAP2_VISIBLE record.
86238592
*
@@ -9722,15 +9691,15 @@ heap2_redo(XLogReaderState *record)
97229691

97239692
switch (info&XLOG_HEAP_OPMASK)
97249693
{
9725-
caseXLOG_HEAP2_CLEAN:
9726-
heap_xlog_clean(record);
9694+
caseXLOG_HEAP2_PRUNE:
9695+
heap_xlog_prune(record);
9696+
break;
9697+
caseXLOG_HEAP2_VACUUM:
9698+
heap_xlog_vacuum(record);
97279699
break;
97289700
caseXLOG_HEAP2_FREEZE_PAGE:
97299701
heap_xlog_freeze_page(record);
97309702
break;
9731-
caseXLOG_HEAP2_CLEANUP_INFO:
9732-
heap_xlog_cleanup_info(record);
9733-
break;
97349703
caseXLOG_HEAP2_VISIBLE:
97359704
heap_xlog_visible(record);
97369705
break;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp