Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit687f2cd

Browse files
Avoid pin scan for replay of XLOG_BTREE_VACUUM
Replay of XLOG_BTREE_VACUUM during Hot Standby was previously thought to requirecomplex interlocking that matched the requirements on the master. This requiredan O(N) operation that became a significant problem with large indexes, causingreplication delays of seconds or in some cases minutes while theXLOG_BTREE_VACUUM was replayed.This commit skips the “pin scan” that was previously required, by observing indetail when and how it is safe to do so, with full documentation. The pin scanis skipped only in replay; the VACUUM code path on master is not touched here.The current commit still performs the pin scan for toast indexes, though thiscan also be avoided if we recheck scans on toast indexes. Later patch willaddress this.No tests included. Manual tests using an additional patch to view WAL recordsand their timing have shown the change in WAL records and their handling hassuccessfully reduced replication delay.
1 parent4631721 commit687f2cd

File tree

5 files changed

+67
-6
lines changed

5 files changed

+67
-6
lines changed

‎src/backend/access/nbtree/README

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -520,6 +520,30 @@ normal running after recovery has completed. This is a key capability
520520
because it allows running applications to continue while the standby
521521
changes state into a normally running server.
522522

523+
The interlocking required to avoid returning incorrect results from
524+
MVCC scans is not required on standby nodes. That is because
525+
HeapTupleSatisfiesUpdate(), HeapTupleSatisfiesSelf(),
526+
HeapTupleSatisfiesDirty() and HeapTupleSatisfiesVacuum() are only
527+
ever used during write transactions, which cannot exist on the standby.
528+
This leaves HeapTupleSatisfiesMVCC() and HeapTupleSatisfiesToast(), so
529+
HeapTupleSatisfiesToast() is the only non-MVCC scan type used on standbys.
530+
There is one minor exception, which is that the optimizer sometimes
531+
looks at the boundaries of value ranges using SnapshotDirty, which
532+
could result in returning a newer value for query statistics; this
533+
would affect the query plan in rare cases, but not the correctness.
534+
The risk window is small since the stats look at the min and max values
535+
in the index, so the scan retrieves a tid then immediately uses it
536+
to look in the heap. It is unlikely that the tid could have been
537+
deleted, vacuumed and re-inserted in the time taken to look in the heap
538+
via direct tid access. So we ignore that scan type as a problem.
539+
This means if we re-check the results of any scan of a toast index we
540+
will be able to completely avoid performing the "pin scan" operation
541+
during replay of VACUUM WAL records.
542+
543+
XXX FIXME: Toast re-checks are not yet added, so we still perform the
544+
pin scan when replaying vacuum records of toast indexes.
545+
546+
523547
Other Things That Are Handy to Know
524548
-----------------------------------
525549

‎src/backend/access/nbtree/nbtree.c

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
#include"access/relscan.h"
2323
#include"access/xlog.h"
2424
#include"catalog/index.h"
25+
#include"catalog/pg_namespace.h"
2526
#include"commands/vacuum.h"
2627
#include"storage/indexfsm.h"
2728
#include"storage/ipc.h"
@@ -823,6 +824,11 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
823824
}
824825

825826
/*
827+
* Check to see if we need to issue one final WAL record for this index,
828+
* which may be needed for correctness on a hot standby node when
829+
* non-MVCC index scans could take place. This now only occurs when we
830+
* perform a TOAST scan, so only occurs for TOAST indexes.
831+
*
826832
* If the WAL is replayed in hot standby, the replay process needs to get
827833
* cleanup locks on all index leaf pages, just as we've been doing here.
828834
* However, we won't issue any WAL records about pages that have no items
@@ -833,6 +839,7 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
833839
* against the last leaf page in the index, if that one wasn't vacuumed.
834840
*/
835841
if (XLogStandbyInfoActive()&&
842+
rel->rd_rel->relnamespace==PG_TOAST_NAMESPACE&&
836843
vstate.lastBlockVacuumed<vstate.lastBlockLocked)
837844
{
838845
Bufferbuf;
@@ -1031,6 +1038,20 @@ btvacuumpage(BTVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
10311038
*/
10321039
if (ndeletable>0)
10331040
{
1041+
BlockNumberlastBlockVacuumed=InvalidBlockNumber;
1042+
1043+
/*
1044+
* We may need to record the lastBlockVacuumed for use when
1045+
* non-MVCC scans might be performed on the index on a
1046+
* hot standby. See explanation in btree_xlog_vacuum().
1047+
*
1048+
* On a hot standby, a non-MVCC scan can only take place
1049+
* when we access a Toast Index, so we need only record
1050+
* the lastBlockVacuumed if we are vacuuming a Toast Index.
1051+
*/
1052+
if (rel->rd_rel->relnamespace==PG_TOAST_NAMESPACE)
1053+
lastBlockVacuumed=vstate->lastBlockVacuumed;
1054+
10341055
/*
10351056
* Notice that the issued XLOG_BTREE_VACUUM WAL record includes an
10361057
* instruction to the replay code to get cleanup lock on all pages
@@ -1043,7 +1064,7 @@ btvacuumpage(BTVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
10431064
* that.
10441065
*/
10451066
_bt_delitems_vacuum(rel,buf,deletable,ndeletable,
1046-
vstate->lastBlockVacuumed);
1067+
lastBlockVacuumed);
10471068

10481069
/*
10491070
* Remember highest leaf page number we've issued a

‎src/backend/access/nbtree/nbtxlog.c

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -391,6 +391,19 @@ btree_xlog_vacuum(XLogReaderState *record)
391391
BTPageOpaqueopaque;
392392

393393
/*
394+
* If we are running non-MVCC scans using this index we need to do some
395+
* additional work to ensure correctness, which is known as a "pin scan"
396+
* described in more detail in next paragraphs. We used to do the extra
397+
* work in all cases, whereas we now avoid that work except when the index
398+
* is a toast index, since toast scans aren't fully MVCC compliant.
399+
* If lastBlockVacuumed is set to InvalidBlockNumber then we skip the
400+
* additional work required for the pin scan.
401+
*
402+
* Avoiding this extra work is important since it requires us to touch
403+
* every page in the index, so is an O(N) operation. Worse, it is an
404+
* operation performed in the foreground during redo, so it delays
405+
* replication directly.
406+
*
394407
* If queries might be active then we need to ensure every leaf page is
395408
* unpinned between the lastBlockVacuumed and the current block, if there
396409
* are any. This prevents replay of the VACUUM from reaching the stage of
@@ -412,7 +425,7 @@ btree_xlog_vacuum(XLogReaderState *record)
412425
* isn't yet consistent; so we need not fear reading still-corrupt blocks
413426
* here during crash recovery.
414427
*/
415-
if (HotStandbyActiveInReplay())
428+
if (HotStandbyActiveInReplay()&&BlockNumberIsValid(xlrec->lastBlockVacuumed))
416429
{
417430
RelFileNodethisrnode;
418431
BlockNumberthisblkno;
@@ -433,7 +446,8 @@ btree_xlog_vacuum(XLogReaderState *record)
433446
* XXX we don't actually need to read the block, we just need to
434447
* confirm it is unpinned. If we had a special call into the
435448
* buffer manager we could optimise this so that if the block is
436-
* not in shared_buffers we confirm it as unpinned.
449+
* not in shared_buffers we confirm it as unpinned. Optimizing
450+
* this is now moot, since in most cases we avoid the scan.
437451
*/
438452
buffer=XLogReadBufferExtended(thisrnode,MAIN_FORKNUM,blkno,
439453
RBM_NORMAL_NO_LOG);

‎src/backend/access/rmgrdesc/nbtdesc.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ btree_desc(StringInfo buf, XLogReaderState *record)
4848
{
4949
xl_btree_vacuum*xlrec= (xl_btree_vacuum*)rec;
5050

51-
appendStringInfo(buf,"lastBlockVacuumed %u",
51+
appendStringInfo(buf,"lastBlockVacuumed %d",
5252
xlrec->lastBlockVacuumed);
5353
break;
5454
}

‎src/include/access/nbtree.h

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -331,8 +331,10 @@ typedef struct xl_btree_reuse_page
331331
* The WAL record can represent deletion of any number of index tuples on a
332332
* single index page when executed by VACUUM.
333333
*
334-
* The correctness requirement for applying these changes during recovery is
335-
* that we must do one of these two things for every block in the index:
334+
* For MVCC scans, lastBlockVacuumed will be set to InvalidBlockNumber.
335+
* For a non-MVCC index scans there is an additional correctness requirement
336+
* for applying these changes during recovery, which is that we must do one
337+
* of these two things for every block in the index:
336338
** lock the block for cleanup and apply any required changes
337339
** EnsureBlockUnpinned()
338340
* The purpose of this is to ensure that no index scans started before we

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp