Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitd3f48df

Browse files
committed
Further reduce memory footprint of CLOBBER_CACHE_ALWAYS testing.
Some buildfarm members using CLOBBER_CACHE_ALWAYS have been having OOMproblems of late. Commit2455ab4 addressed this problem by recoveringspace transiently used within RelationBuildPartitionDesc, but it turnsout that leaves quite a lot on the table, because other subroutines ofRelationBuildDesc also leak memory like mad. Let's move the temp-contextmanagement into RelationBuildDesc so that leakage from the othersubroutines is also recovered.I examined this issue by arranging for postgres.c to dump the size ofMessageContext just before resetting it in each command cycle, andthen running the update.sql regression test (which is one of the twothat are seeing buildfarm OOMs) with and without CLOBBER_CACHE_ALWAYS.Before2455ab4, the peak space usage with CCA was as much as 250MB.That patch got it down to ~80MB, but with this patch it's about 0.5MB,and indeed the space usage now seems nearly indistinguishable from anon-CCA build.RelationBuildDesc's traditional behavior of not worrying about leakingtransient data is of many years' standing, so I'm pretty hesitant tochange that without more evidence that it'd be useful in a normal build.(So far as I can see, non-CCA memory consumption is about the same withor without this change, whuch if anything suggests that it isn't useful.)Hence, configure the patch so that we recover space only whenCLOBBER_CACHE_ALWAYS or CLOBBER_CACHE_RECURSIVELY is defined. However,that choice can be overridden at compile time, in case somebody wouldlike to do some performance testing and try to develop evidence forchanging that decision.It's possible that we ought to back-patch this change, but in theabsence of back-branch OOM problems in the buildfarm, I'm not ina hurry to do that.Discussion:https://postgr.es/m/CA+TgmoY3bRmGB6-DUnoVy5fJoreiBJ43rwMrQRCdPXuKt4Ykaw@mail.gmail.com
1 parentaefcc2b commitd3f48df

File tree

2 files changed

+52
-19
lines changed

2 files changed

+52
-19
lines changed

‎src/backend/partitioning/partdesc.c

Lines changed: 4 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -67,19 +67,8 @@ RelationBuildPartitionDesc(Relation rel)
6767
nparts;
6868
PartitionKeykey=RelationGetPartitionKey(rel);
6969
MemoryContextoldcxt;
70-
MemoryContextrbcontext;
7170
int*mapping;
7271

73-
/*
74-
* While building the partition descriptor, we create various temporary
75-
* data structures; in CLOBBER_CACHE_ALWAYS mode, at least, it's important
76-
* not to leak them, since this can get called a lot.
77-
*/
78-
rbcontext=AllocSetContextCreate(CurrentMemoryContext,
79-
"RelationBuildPartitionDesc",
80-
ALLOCSET_DEFAULT_SIZES);
81-
oldcxt=MemoryContextSwitchTo(rbcontext);
82-
8372
/*
8473
* Get partition oids from pg_inherits. This uses a single snapshot to
8574
* fetch the list of children, so while more children may be getting
@@ -197,14 +186,15 @@ RelationBuildPartitionDesc(Relation rel)
197186
/* If there are no partitions, the rest of the partdesc can stay zero */
198187
if (nparts>0)
199188
{
200-
/* Create PartitionBoundInfo, using thetemporary context. */
189+
/* Create PartitionBoundInfo, using thecaller's context. */
201190
boundinfo=partition_bounds_create(boundspecs,nparts,key,&mapping);
202191

203192
/* Now copy all info into relcache's partdesc. */
204-
MemoryContextSwitchTo(rel->rd_pdcxt);
193+
oldcxt=MemoryContextSwitchTo(rel->rd_pdcxt);
205194
partdesc->boundinfo=partition_bounds_copy(boundinfo,key);
206195
partdesc->oids= (Oid*)palloc(nparts*sizeof(Oid));
207196
partdesc->is_leaf= (bool*)palloc(nparts*sizeof(bool));
197+
MemoryContextSwitchTo(oldcxt);
208198

209199
/*
210200
* Assign OIDs from the original array into mapped indexes of the
@@ -214,9 +204,8 @@ RelationBuildPartitionDesc(Relation rel)
214204
*
215205
* Also record leaf-ness of each partition. For this we use
216206
* get_rel_relkind() which may leak memory, so be sure to run it in
217-
* thetemporary context.
207+
* thecaller's context.
218208
*/
219-
MemoryContextSwitchTo(rbcontext);
220209
for (i=0;i<nparts;i++)
221210
{
222211
intindex=mapping[i];
@@ -228,10 +217,6 @@ RelationBuildPartitionDesc(Relation rel)
228217
}
229218

230219
rel->rd_partdesc=partdesc;
231-
232-
/* Return to caller's context, and blow away the temporary context. */
233-
MemoryContextSwitchTo(oldcxt);
234-
MemoryContextDelete(rbcontext);
235220
}
236221

237222
/*

‎src/backend/utils/cache/relcache.c

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,19 @@
9393

9494
#defineRELCACHE_INIT_FILEMAGIC0x573266/* version ID value */
9595

96+
/*
97+
* Default policy for whether to apply RECOVER_RELATION_BUILD_MEMORY:
98+
* do so in clobber-cache builds but not otherwise. This choice can be
99+
* overridden at compile time with -DRECOVER_RELATION_BUILD_MEMORY=1 or =0.
100+
*/
101+
#ifndefRECOVER_RELATION_BUILD_MEMORY
102+
#if defined(CLOBBER_CACHE_ALWAYS)|| defined(CLOBBER_CACHE_RECURSIVELY)
103+
#defineRECOVER_RELATION_BUILD_MEMORY 1
104+
#else
105+
#defineRECOVER_RELATION_BUILD_MEMORY 0
106+
#endif
107+
#endif
108+
96109
/*
97110
*hardcoded tuple descriptors, contents generated by genbki.pl
98111
*/
@@ -1014,6 +1027,28 @@ RelationBuildDesc(Oid targetRelId, bool insertIt)
10141027
HeapTuplepg_class_tuple;
10151028
Form_pg_classrelp;
10161029

1030+
/*
1031+
* This function and its subroutines can allocate a good deal of transient
1032+
* data in CurrentMemoryContext. Traditionally we've just leaked that
1033+
* data, reasoning that the caller's context is at worst of transaction
1034+
* scope, and relcache loads shouldn't happen so often that it's essential
1035+
* to recover transient data before end of statement/transaction. However
1036+
* that's definitely not true in clobber-cache test builds, and perhaps
1037+
* it's not true in other cases. If RECOVER_RELATION_BUILD_MEMORY is not
1038+
* zero, arrange to allocate the junk in a temporary context that we'll
1039+
* free before returning. Make it a child of caller's context so that it
1040+
* will get cleaned up appropriately if we error out partway through.
1041+
*/
1042+
#ifRECOVER_RELATION_BUILD_MEMORY
1043+
MemoryContexttmpcxt;
1044+
MemoryContextoldcxt;
1045+
1046+
tmpcxt=AllocSetContextCreate(CurrentMemoryContext,
1047+
"RelationBuildDesc workspace",
1048+
ALLOCSET_DEFAULT_SIZES);
1049+
oldcxt=MemoryContextSwitchTo(tmpcxt);
1050+
#endif
1051+
10171052
/*
10181053
* find the tuple in pg_class corresponding to the given relation id
10191054
*/
@@ -1023,7 +1058,14 @@ RelationBuildDesc(Oid targetRelId, bool insertIt)
10231058
* if no such tuple exists, return NULL
10241059
*/
10251060
if (!HeapTupleIsValid(pg_class_tuple))
1061+
{
1062+
#ifRECOVER_RELATION_BUILD_MEMORY
1063+
/* Return to caller's context, and blow away the temporary context */
1064+
MemoryContextSwitchTo(oldcxt);
1065+
MemoryContextDelete(tmpcxt);
1066+
#endif
10261067
returnNULL;
1068+
}
10271069

10281070
/*
10291071
* get information from the pg_class_tuple
@@ -1203,6 +1245,12 @@ RelationBuildDesc(Oid targetRelId, bool insertIt)
12031245
/* It's fully valid */
12041246
relation->rd_isvalid= true;
12051247

1248+
#ifRECOVER_RELATION_BUILD_MEMORY
1249+
/* Return to caller's context, and blow away the temporary context */
1250+
MemoryContextSwitchTo(oldcxt);
1251+
MemoryContextDelete(tmpcxt);
1252+
#endif
1253+
12061254
returnrelation;
12071255
}
12081256

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp