Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit4e514c6

Browse files
author
Amit Kapila
committed
Delete empty pages in each pass during GIST VACUUM.
Earlier, we use to postpone deleting empty pages till the second stage ofvacuum to amortize the cost of scanning internal pages. However, that cansometimes (say vacuum is canceled or errored between first and secondstage) delay the pages to be recycled.Another thing is that to facilitate deleting empty pages in the secondstage, we need to share the information about internal and empty pagesbetween different stages of vacuum. It will be quite tricky to share thisinformation via DSM which is required for the upcoming parallel vacuumpatch.Also, it will bring the logic to reclaim deleted pages closer to nbtreewhere we delete empty pages in each pass.Overall, the advantages of deleting empty pages in each pass outweigh theadvantages of postponing the same.Author: Dilip Kumar, with changes by Amit KapilaReviewed-by: Sawada Masahiko and Amit KapilaDiscussion:https://postgr.es/m/CAA4eK1LGr+MN0xHZpJ2dfS8QNQ1a_aROKowZB+MPNep8FVtwAA@mail.gmail.com
1 parenteae056c commit4e514c6

File tree

2 files changed

+78
-105
lines changed

2 files changed

+78
-105
lines changed

‎src/backend/access/gist/README

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -429,18 +429,17 @@ splits during searches, we don't need a "vacuum cycle ID" concept for that
429429
like B-tree does.
430430

431431
While we scan all the pages, we also make note of any completely empty leaf
432-
pages. We will try to unlink them from the tree in the second stage. We also
433-
record the block numbers of all internal pages; they are needed in the second
434-
stage, to locate parents of the empty pages.
435-
436-
In the second stage, we try to unlink any empty leaf pages from the tree, so
437-
that their space can be reused. In order to delete an empty page, its
438-
downlink must be removed from the parent. We scan all the internal pages,
439-
whose block numbers we memorized in the first stage, and look for downlinks
440-
to pages that we have memorized as being empty. Whenever we find one, we
441-
acquire a lock on the parent and child page, re-check that the child page is
442-
still empty. Then, we remove the downlink and mark the child as deleted, and
443-
release the locks.
432+
pages. We will try to unlink them from the tree after the scan. We also record
433+
the block numbers of all internal pages; they are needed to locate parents of
434+
the empty pages while unlinking them.
435+
436+
We try to unlink any empty leaf pages from the tree, so that their space can
437+
be reused. In order to delete an empty page, its downlink must be removed from
438+
the parent. We scan all the internal pages, whose block numbers we memorized
439+
in the first stage, and look for downlinks to pages that we have memorized as
440+
being empty. Whenever we find one, we acquire a lock on the parent and child
441+
page, re-check that the child page is still empty. Then, we remove the
442+
downlink and mark the child as deleted, and release the locks.
444443

445444
The insertion algorithm would get confused, if an internal page was completely
446445
empty. So we never delete the last child of an internal page, even if it's

‎src/backend/access/gist/gistvacuum.c

Lines changed: 67 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -24,74 +24,48 @@
2424
#include"storage/lmgr.h"
2525
#include"utils/memutils.h"
2626

27-
/*
28-
* State kept across vacuum stages.
29-
*/
27+
/* Working state needed by gistbulkdelete */
3028
typedefstruct
3129
{
32-
IndexBulkDeleteResultstats;/* must be first */
30+
IndexVacuumInfo*info;
31+
IndexBulkDeleteResult*stats;
32+
IndexBulkDeleteCallbackcallback;
33+
void*callback_state;
34+
GistNSNstartNSN;
3335

3436
/*
35-
* These are used to memorize all internal and empty leaf pages in the 1st
36-
* vacuum stage. They are used in the 2nd stage, to delete all the empty
37-
* pages.
37+
* These are used to memorize all internal and empty leaf pages. They are
38+
* used for deleting all the empty pages.
3839
*/
3940
IntegerSet*internal_page_set;
4041
IntegerSet*empty_leaf_set;
4142
MemoryContextpage_set_context;
42-
}GistBulkDeleteResult;
43-
44-
/* Working state needed by gistbulkdelete */
45-
typedefstruct
46-
{
47-
IndexVacuumInfo*info;
48-
GistBulkDeleteResult*stats;
49-
IndexBulkDeleteCallbackcallback;
50-
void*callback_state;
51-
GistNSNstartNSN;
5243
}GistVacState;
5344

54-
staticvoidgistvacuumscan(IndexVacuumInfo*info,GistBulkDeleteResult*stats,
45+
staticvoidgistvacuumscan(IndexVacuumInfo*info,IndexBulkDeleteResult*stats,
5546
IndexBulkDeleteCallbackcallback,void*callback_state);
5647
staticvoidgistvacuumpage(GistVacState*vstate,BlockNumberblkno,
5748
BlockNumberorig_blkno);
5849
staticvoidgistvacuum_delete_empty_pages(IndexVacuumInfo*info,
59-
GistBulkDeleteResult*stats);
60-
staticboolgistdeletepage(IndexVacuumInfo*info,GistBulkDeleteResult*stats,
50+
GistVacState*vstate);
51+
staticboolgistdeletepage(IndexVacuumInfo*info,IndexBulkDeleteResult*stats,
6152
Bufferbuffer,OffsetNumberdownlink,
6253
BufferleafBuffer);
6354

64-
/* allocate the 'stats' struct that's kept over vacuum stages */
65-
staticGistBulkDeleteResult*
66-
create_GistBulkDeleteResult(void)
67-
{
68-
GistBulkDeleteResult*gist_stats;
69-
70-
gist_stats= (GistBulkDeleteResult*)palloc0(sizeof(GistBulkDeleteResult));
71-
gist_stats->page_set_context=
72-
GenerationContextCreate(CurrentMemoryContext,
73-
"GiST VACUUM page set context",
74-
16*1024);
75-
76-
returngist_stats;
77-
}
78-
7955
/*
8056
* VACUUM bulkdelete stage: remove index entries.
8157
*/
8258
IndexBulkDeleteResult*
8359
gistbulkdelete(IndexVacuumInfo*info,IndexBulkDeleteResult*stats,
8460
IndexBulkDeleteCallbackcallback,void*callback_state)
8561
{
86-
GistBulkDeleteResult*gist_stats= (GistBulkDeleteResult*)stats;
87-
8862
/* allocate stats if first time through, else re-use existing struct */
89-
if (gist_stats==NULL)
90-
gist_stats=create_GistBulkDeleteResult();
63+
if (stats==NULL)
64+
stats=(IndexBulkDeleteResult*)palloc0(sizeof(IndexBulkDeleteResult));
9165

92-
gistvacuumscan(info,gist_stats,callback,callback_state);
66+
gistvacuumscan(info,stats,callback,callback_state);
9367

94-
return(IndexBulkDeleteResult*)gist_stats;
68+
returnstats;
9569
}
9670

9771
/*
@@ -100,8 +74,6 @@ gistbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
10074
IndexBulkDeleteResult*
10175
gistvacuumcleanup(IndexVacuumInfo*info,IndexBulkDeleteResult*stats)
10276
{
103-
GistBulkDeleteResult*gist_stats= (GistBulkDeleteResult*)stats;
104-
10577
/* No-op in ANALYZE ONLY mode */
10678
if (info->analyze_only)
10779
returnstats;
@@ -111,24 +83,12 @@ gistvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
11183
* stats from the latest gistbulkdelete call. If it wasn't called, we
11284
* still need to do a pass over the index, to obtain index statistics.
11385
*/
114-
if (gist_stats==NULL)
86+
if (stats==NULL)
11587
{
116-
gist_stats=create_GistBulkDeleteResult();
117-
gistvacuumscan(info,gist_stats,NULL,NULL);
88+
stats=(IndexBulkDeleteResult*)palloc0(sizeof(IndexBulkDeleteResult));
89+
gistvacuumscan(info,stats,NULL,NULL);
11890
}
11991

120-
/*
121-
* If we saw any empty pages, try to unlink them from the tree so that
122-
* they can be reused.
123-
*/
124-
gistvacuum_delete_empty_pages(info,gist_stats);
125-
126-
/* we don't need the internal and empty page sets anymore */
127-
MemoryContextDelete(gist_stats->page_set_context);
128-
gist_stats->page_set_context=NULL;
129-
gist_stats->internal_page_set=NULL;
130-
gist_stats->empty_leaf_set=NULL;
131-
13292
/*
13393
* It's quite possible for us to be fooled by concurrent page splits into
13494
* double-counting some index tuples, so disbelieve any total that exceeds
@@ -137,11 +97,11 @@ gistvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
13797
*/
13898
if (!info->estimated_count)
13999
{
140-
if (gist_stats->stats.num_index_tuples>info->num_heap_tuples)
141-
gist_stats->stats.num_index_tuples=info->num_heap_tuples;
100+
if (stats->num_index_tuples>info->num_heap_tuples)
101+
stats->num_index_tuples=info->num_heap_tuples;
142102
}
143103

144-
return(IndexBulkDeleteResult*)gist_stats;
104+
returnstats;
145105
}
146106

147107
/*
@@ -153,15 +113,16 @@ gistvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
153113
* occurred).
154114
*
155115
* This also makes note of any empty leaf pages, as well as all internal
156-
* pages. The second stage, gistvacuum_delete_empty_pages(), needs that
157-
* information. Any deleted pages are added directly to the free space map.
158-
* (They should've been added there when they were originally deleted, already,
159-
* but it's possible that the FSM was lost at a crash, for example.)
116+
* pages while looping over all index pages. After scanning all the pages, we
117+
* remove the empty pages so that they can be reused. Any deleted pages are
118+
* added directly to the free space map. (They should've been added there
119+
* when they were originally deleted, already, but it's possible that the FSM
120+
* was lost at a crash, for example.)
160121
*
161122
* The caller is responsible for initially allocating/zeroing a stats struct.
162123
*/
163124
staticvoid
164-
gistvacuumscan(IndexVacuumInfo*info,GistBulkDeleteResult*stats,
125+
gistvacuumscan(IndexVacuumInfo*info,IndexBulkDeleteResult*stats,
165126
IndexBulkDeleteCallbackcallback,void*callback_state)
166127
{
167128
Relationrel=info->index;
@@ -175,21 +136,23 @@ gistvacuumscan(IndexVacuumInfo *info, GistBulkDeleteResult *stats,
175136
* Reset counts that will be incremented during the scan; needed in case
176137
* of multiple scans during a single VACUUM command.
177138
*/
178-
stats->stats.estimated_count= false;
179-
stats->stats.num_index_tuples=0;
180-
stats->stats.pages_deleted=0;
181-
stats->stats.pages_free=0;
182-
MemoryContextReset(stats->page_set_context);
139+
stats->estimated_count= false;
140+
stats->num_index_tuples=0;
141+
stats->pages_deleted=0;
142+
stats->pages_free=0;
183143

184144
/*
185145
* Create the integer sets to remember all the internal and the empty leaf
186146
* pages in page_set_context. Internally, the integer set will remember
187147
* this context so that the subsequent allocations for these integer sets
188148
* will be done from the same context.
189149
*/
190-
oldctx=MemoryContextSwitchTo(stats->page_set_context);
191-
stats->internal_page_set=intset_create();
192-
stats->empty_leaf_set=intset_create();
150+
vstate.page_set_context=GenerationContextCreate(CurrentMemoryContext,
151+
"GiST VACUUM page set context",
152+
16*1024);
153+
oldctx=MemoryContextSwitchTo(vstate.page_set_context);
154+
vstate.internal_page_set=intset_create();
155+
vstate.empty_leaf_set=intset_create();
193156
MemoryContextSwitchTo(oldctx);
194157

195158
/* Set up info to pass down to gistvacuumpage */
@@ -257,11 +220,23 @@ gistvacuumscan(IndexVacuumInfo *info, GistBulkDeleteResult *stats,
257220
* Note that if no recyclable pages exist, we don't bother vacuuming the
258221
* FSM at all.
259222
*/
260-
if (stats->stats.pages_free>0)
223+
if (stats->pages_free>0)
261224
IndexFreeSpaceMapVacuum(rel);
262225

263226
/* update statistics */
264-
stats->stats.num_pages=num_pages;
227+
stats->num_pages=num_pages;
228+
229+
/*
230+
* If we saw any empty pages, try to unlink them from the tree so that
231+
* they can be reused.
232+
*/
233+
gistvacuum_delete_empty_pages(info,&vstate);
234+
235+
/* we don't need the internal and empty page sets anymore */
236+
MemoryContextDelete(vstate.page_set_context);
237+
vstate.page_set_context=NULL;
238+
vstate.internal_page_set=NULL;
239+
vstate.empty_leaf_set=NULL;
265240
}
266241

267242
/*
@@ -278,7 +253,6 @@ gistvacuumscan(IndexVacuumInfo *info, GistBulkDeleteResult *stats,
278253
staticvoid
279254
gistvacuumpage(GistVacState*vstate,BlockNumberblkno,BlockNumberorig_blkno)
280255
{
281-
GistBulkDeleteResult*stats=vstate->stats;
282256
IndexVacuumInfo*info=vstate->info;
283257
IndexBulkDeleteCallbackcallback=vstate->callback;
284258
void*callback_state=vstate->callback_state;
@@ -307,13 +281,13 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
307281
{
308282
/* Okay to recycle this page */
309283
RecordFreeIndexPage(rel,blkno);
310-
stats->stats.pages_free++;
311-
stats->stats.pages_deleted++;
284+
vstate->stats->pages_free++;
285+
vstate->stats->pages_deleted++;
312286
}
313287
elseif (GistPageIsDeleted(page))
314288
{
315289
/* Already deleted, but can't recycle yet */
316-
stats->stats.pages_deleted++;
290+
vstate->stats->pages_deleted++;
317291
}
318292
elseif (GistPageIsLeaf(page))
319293
{
@@ -388,7 +362,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
388362

389363
END_CRIT_SECTION();
390364

391-
stats->stats.tuples_removed+=ntodelete;
365+
vstate->stats->tuples_removed+=ntodelete;
392366
/* must recompute maxoff */
393367
maxoff=PageGetMaxOffsetNumber(page);
394368
}
@@ -405,10 +379,10 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
405379
* it up.
406380
*/
407381
if (blkno==orig_blkno)
408-
intset_add_member(stats->empty_leaf_set,blkno);
382+
intset_add_member(vstate->empty_leaf_set,blkno);
409383
}
410384
else
411-
stats->stats.num_index_tuples+=nremain;
385+
vstate->stats->num_index_tuples+=nremain;
412386
}
413387
else
414388
{
@@ -443,7 +417,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
443417
* parents of empty leaf pages.
444418
*/
445419
if (blkno==orig_blkno)
446-
intset_add_member(stats->internal_page_set,blkno);
420+
intset_add_member(vstate->internal_page_set,blkno);
447421
}
448422

449423
UnlockReleaseBuffer(buffer);
@@ -466,7 +440,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
466440
* Scan all internal pages, and try to delete their empty child pages.
467441
*/
468442
staticvoid
469-
gistvacuum_delete_empty_pages(IndexVacuumInfo*info,GistBulkDeleteResult*stats)
443+
gistvacuum_delete_empty_pages(IndexVacuumInfo*info,GistVacState*vstate)
470444
{
471445
Relationrel=info->index;
472446
BlockNumberempty_pages_remaining;
@@ -475,10 +449,10 @@ gistvacuum_delete_empty_pages(IndexVacuumInfo *info, GistBulkDeleteResult *stats
475449
/*
476450
* Rescan all inner pages to find those that have empty child pages.
477451
*/
478-
empty_pages_remaining=intset_num_entries(stats->empty_leaf_set);
479-
intset_begin_iterate(stats->internal_page_set);
452+
empty_pages_remaining=intset_num_entries(vstate->empty_leaf_set);
453+
intset_begin_iterate(vstate->internal_page_set);
480454
while (empty_pages_remaining>0&&
481-
intset_iterate_next(stats->internal_page_set,&blkno))
455+
intset_iterate_next(vstate->internal_page_set,&blkno))
482456
{
483457
Bufferbuffer;
484458
Pagepage;
@@ -521,7 +495,7 @@ gistvacuum_delete_empty_pages(IndexVacuumInfo *info, GistBulkDeleteResult *stats
521495
BlockNumberleafblk;
522496

523497
leafblk=ItemPointerGetBlockNumber(&(idxtuple->t_tid));
524-
if (intset_is_member(stats->empty_leaf_set,leafblk))
498+
if (intset_is_member(vstate->empty_leaf_set,leafblk))
525499
{
526500
leafs_to_delete[ntodelete]=leafblk;
527501
todelete[ntodelete++]=off;
@@ -561,7 +535,7 @@ gistvacuum_delete_empty_pages(IndexVacuumInfo *info, GistBulkDeleteResult *stats
561535
gistcheckpage(rel,leafbuf);
562536

563537
LockBuffer(buffer,GIST_EXCLUSIVE);
564-
if (gistdeletepage(info,stats,
538+
if (gistdeletepage(info,vstate->stats,
565539
buffer,todelete[i]-deleted,
566540
leafbuf))
567541
deleted++;
@@ -573,7 +547,7 @@ gistvacuum_delete_empty_pages(IndexVacuumInfo *info, GistBulkDeleteResult *stats
573547
ReleaseBuffer(buffer);
574548

575549
/* update stats */
576-
stats->stats.pages_removed+=deleted;
550+
vstate->stats->pages_removed+=deleted;
577551

578552
/*
579553
* We can stop the scan as soon as we have seen the downlinks, even if
@@ -596,7 +570,7 @@ gistvacuum_delete_empty_pages(IndexVacuumInfo *info, GistBulkDeleteResult *stats
596570
* prevented it.
597571
*/
598572
staticbool
599-
gistdeletepage(IndexVacuumInfo*info,GistBulkDeleteResult*stats,
573+
gistdeletepage(IndexVacuumInfo*info,IndexBulkDeleteResult*stats,
600574
BufferparentBuffer,OffsetNumberdownlink,
601575
BufferleafBuffer)
602576
{
@@ -665,7 +639,7 @@ gistdeletepage(IndexVacuumInfo *info, GistBulkDeleteResult *stats,
665639
/* mark the page as deleted */
666640
MarkBufferDirty(leafBuffer);
667641
GistPageSetDeleted(leafPage,txid);
668-
stats->stats.pages_deleted++;
642+
stats->pages_deleted++;
669643

670644
/* remove the downlink from the parent */
671645
MarkBufferDirty(parentBuffer);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp