Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit558a916

Browse files
committed
Compute XID horizon for page level index vacuum on primary.
Previously the xid horizon was only computed during WAL replay. Thathad two major problems:1) It relied on knowing what the table pointed to looks like. That was easy enough before the introducing of tableam (we knew it had to be heap, although some trickery around logging the heap relfilenodes was required). But to properly handle table AMs we need per-database catalog access to look up the AM handler, which recovery doesn't allow.2) Not knowing the xid horizon also makes it hard to support logical decoding on standbys. When on a catalog table, we need to be able to conflict with slots that have an xid horizon that's too old. But computing the horizon by visiting the heap only works once consistency is reached, but we always need to be able to detect conflicts.There's also a secondary problem, in that the current method performsredundant work on every standby. But that's counterbalanced bypotentially computing the value when not necessary (either becausethere's no standby, or because there's no connected backends).Solve 1) and 2) by moving computation of the xid horizon to theprimary and by involving tableam in the computation of the horizon.To address the potentially increased overhead, increase the efficiencyof the xid horizon computation for heap by sorting the tids, andeliminating redundant buffer accesses. When prefetching is available,additionally perform prefetching of buffers. As this is more of amaintenance task, rather than something routinely done in every readonly query, we add an arbitrary 10 to the effective concurrency -thereby using IO concurrency, when not globally enabled. That'spossibly not the perfect formula, but seems good enough for now.Bumps WAL format, as latestRemovedXid is now part of the records, andthe heap's relfilenode isn't anymore.Author: Andres Freund, Amit Khandekar, Robert HaasReviewed-By: Robert HaasDiscussion:https://postgr.es/m/20181212204154.nsxf3gzqv3gesl32@alap3.anarazel.dehttps://postgr.es/m/20181214014235.dal5ogljs3bmlq44@alap3.anarazel.dehttps://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
1 parent126d631 commit558a916

File tree

16 files changed

+316
-321
lines changed

16 files changed

+316
-321
lines changed

‎src/backend/access/hash/hash_xlog.c

Lines changed: 1 addition & 152 deletions
Original file line numberDiff line numberDiff line change
@@ -969,155 +969,6 @@ hash_xlog_update_meta_page(XLogReaderState *record)
969969
UnlockReleaseBuffer(metabuf);
970970
}
971971

972-
/*
973-
* Get the latestRemovedXid from the heap pages pointed at by the index
974-
* tuples being deleted. See also btree_xlog_delete_get_latestRemovedXid,
975-
* on which this function is based.
976-
*/
977-
staticTransactionId
978-
hash_xlog_vacuum_get_latestRemovedXid(XLogReaderState*record)
979-
{
980-
xl_hash_vacuum_one_page*xlrec;
981-
OffsetNumber*unused;
982-
Bufferibuffer,
983-
hbuffer;
984-
Pageipage,
985-
hpage;
986-
RelFileNodernode;
987-
BlockNumberblkno;
988-
ItemIdiitemid,
989-
hitemid;
990-
IndexTupleitup;
991-
HeapTupleHeaderhtuphdr;
992-
BlockNumberhblkno;
993-
OffsetNumberhoffnum;
994-
TransactionIdlatestRemovedXid=InvalidTransactionId;
995-
inti;
996-
997-
xlrec= (xl_hash_vacuum_one_page*)XLogRecGetData(record);
998-
999-
/*
1000-
* If there's nothing running on the standby we don't need to derive a
1001-
* full latestRemovedXid value, so use a fast path out of here. This
1002-
* returns InvalidTransactionId, and so will conflict with all HS
1003-
* transactions; but since we just worked out that that's zero people,
1004-
* it's OK.
1005-
*
1006-
* XXX There is a race condition here, which is that a new backend might
1007-
* start just after we look. If so, it cannot need to conflict, but this
1008-
* coding will result in throwing a conflict anyway.
1009-
*/
1010-
if (CountDBBackends(InvalidOid)==0)
1011-
returnlatestRemovedXid;
1012-
1013-
/*
1014-
* Check if WAL replay has reached a consistent database state. If not, we
1015-
* must PANIC. See the definition of
1016-
* btree_xlog_delete_get_latestRemovedXid for more details.
1017-
*/
1018-
if (!reachedConsistency)
1019-
elog(PANIC,"hash_xlog_vacuum_get_latestRemovedXid: cannot operate with inconsistent data");
1020-
1021-
/*
1022-
* Get index page. If the DB is consistent, this should not fail, nor
1023-
* should any of the heap page fetches below. If one does, we return
1024-
* InvalidTransactionId to cancel all HS transactions. That's probably
1025-
* overkill, but it's safe, and certainly better than panicking here.
1026-
*/
1027-
XLogRecGetBlockTag(record,0,&rnode,NULL,&blkno);
1028-
ibuffer=XLogReadBufferExtended(rnode,MAIN_FORKNUM,blkno,RBM_NORMAL);
1029-
1030-
if (!BufferIsValid(ibuffer))
1031-
returnInvalidTransactionId;
1032-
LockBuffer(ibuffer,HASH_READ);
1033-
ipage= (Page)BufferGetPage(ibuffer);
1034-
1035-
/*
1036-
* Loop through the deleted index items to obtain the TransactionId from
1037-
* the heap items they point to.
1038-
*/
1039-
unused= (OffsetNumber*) ((char*)xlrec+SizeOfHashVacuumOnePage);
1040-
1041-
for (i=0;i<xlrec->ntuples;i++)
1042-
{
1043-
/*
1044-
* Identify the index tuple about to be deleted.
1045-
*/
1046-
iitemid=PageGetItemId(ipage,unused[i]);
1047-
itup= (IndexTuple)PageGetItem(ipage,iitemid);
1048-
1049-
/*
1050-
* Locate the heap page that the index tuple points at
1051-
*/
1052-
hblkno=ItemPointerGetBlockNumber(&(itup->t_tid));
1053-
hbuffer=XLogReadBufferExtended(xlrec->hnode,MAIN_FORKNUM,
1054-
hblkno,RBM_NORMAL);
1055-
1056-
if (!BufferIsValid(hbuffer))
1057-
{
1058-
UnlockReleaseBuffer(ibuffer);
1059-
returnInvalidTransactionId;
1060-
}
1061-
LockBuffer(hbuffer,HASH_READ);
1062-
hpage= (Page)BufferGetPage(hbuffer);
1063-
1064-
/*
1065-
* Look up the heap tuple header that the index tuple points at by
1066-
* using the heap node supplied with the xlrec. We can't use
1067-
* heap_fetch, since it uses ReadBuffer rather than XLogReadBuffer.
1068-
* Note that we are not looking at tuple data here, just headers.
1069-
*/
1070-
hoffnum=ItemPointerGetOffsetNumber(&(itup->t_tid));
1071-
hitemid=PageGetItemId(hpage,hoffnum);
1072-
1073-
/*
1074-
* Follow any redirections until we find something useful.
1075-
*/
1076-
while (ItemIdIsRedirected(hitemid))
1077-
{
1078-
hoffnum=ItemIdGetRedirect(hitemid);
1079-
hitemid=PageGetItemId(hpage,hoffnum);
1080-
CHECK_FOR_INTERRUPTS();
1081-
}
1082-
1083-
/*
1084-
* If the heap item has storage, then read the header and use that to
1085-
* set latestRemovedXid.
1086-
*
1087-
* Some LP_DEAD items may not be accessible, so we ignore them.
1088-
*/
1089-
if (ItemIdHasStorage(hitemid))
1090-
{
1091-
htuphdr= (HeapTupleHeader)PageGetItem(hpage,hitemid);
1092-
HeapTupleHeaderAdvanceLatestRemovedXid(htuphdr,&latestRemovedXid);
1093-
}
1094-
elseif (ItemIdIsDead(hitemid))
1095-
{
1096-
/*
1097-
* Conjecture: if hitemid is dead then it had xids before the xids
1098-
* marked on LP_NORMAL items. So we just ignore this item and move
1099-
* onto the next, for the purposes of calculating
1100-
* latestRemovedxids.
1101-
*/
1102-
}
1103-
else
1104-
Assert(!ItemIdIsUsed(hitemid));
1105-
1106-
UnlockReleaseBuffer(hbuffer);
1107-
}
1108-
1109-
UnlockReleaseBuffer(ibuffer);
1110-
1111-
/*
1112-
* If all heap tuples were LP_DEAD then we will be returning
1113-
* InvalidTransactionId here, which avoids conflicts. This matches
1114-
* existing logic which assumes that LP_DEAD tuples must already be older
1115-
* than the latestRemovedXid on the cleanup record that set them as
1116-
* LP_DEAD, hence must already have generated a conflict.
1117-
*/
1118-
returnlatestRemovedXid;
1119-
}
1120-
1121972
/*
1122973
* replay delete operation in hash index to remove
1123974
* tuples marked as DEAD during index tuple insertion.
@@ -1149,12 +1000,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record)
11491000
*/
11501001
if (InHotStandby)
11511002
{
1152-
TransactionIdlatestRemovedXid=
1153-
hash_xlog_vacuum_get_latestRemovedXid(record);
11541003
RelFileNodernode;
11551004

11561005
XLogRecGetBlockTag(record,0,&rnode,NULL,NULL);
1157-
ResolveRecoveryConflictWithSnapshot(latestRemovedXid,rnode);
1006+
ResolveRecoveryConflictWithSnapshot(xldata->latestRemovedXid,rnode);
11581007
}
11591008

11601009
action=XLogReadBufferForRedoExtended(record,0,RBM_NORMAL, true,&buffer);

‎src/backend/access/hash/hashinsert.c

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@
2323
#include"storage/buf_internals.h"
2424
#include"storage/predicate.h"
2525

26-
staticvoid_hash_vacuum_one_page(Relationrel,Buffermetabuf,Bufferbuf,
27-
RelFileNodehnode);
26+
staticvoid_hash_vacuum_one_page(Relationrel,Relationhrel,
27+
Buffermetabuf,Bufferbuf);
2828

2929
/*
3030
*_hash_doinsert() -- Handle insertion of a single index tuple.
@@ -137,7 +137,7 @@ _hash_doinsert(Relation rel, IndexTuple itup, Relation heapRel)
137137

138138
if (IsBufferCleanupOK(buf))
139139
{
140-
_hash_vacuum_one_page(rel,metabuf,buf,heapRel->rd_node);
140+
_hash_vacuum_one_page(rel,heapRel,metabuf,buf);
141141

142142
if (PageGetFreeSpace(page) >=itemsz)
143143
break;/* OK, now we have enough space */
@@ -335,8 +335,7 @@ _hash_pgaddmultitup(Relation rel, Buffer buf, IndexTuple *itups,
335335
*/
336336

337337
staticvoid
338-
_hash_vacuum_one_page(Relationrel,Buffermetabuf,Bufferbuf,
339-
RelFileNodehnode)
338+
_hash_vacuum_one_page(Relationrel,Relationhrel,Buffermetabuf,Bufferbuf)
340339
{
341340
OffsetNumberdeletable[MaxOffsetNumber];
342341
intndeletable=0;
@@ -360,6 +359,12 @@ _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
360359

361360
if (ndeletable>0)
362361
{
362+
TransactionIdlatestRemovedXid;
363+
364+
latestRemovedXid=
365+
index_compute_xid_horizon_for_tuples(rel,hrel,buf,
366+
deletable,ndeletable);
367+
363368
/*
364369
* Write-lock the meta page so that we can decrement tuple count.
365370
*/
@@ -393,7 +398,7 @@ _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
393398
xl_hash_vacuum_one_pagexlrec;
394399
XLogRecPtrrecptr;
395400

396-
xlrec.hnode=hnode;
401+
xlrec.latestRemovedXid=latestRemovedXid;
397402
xlrec.ntuples=ndeletable;
398403

399404
XLogBeginInsert();

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp