Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit118c9bb

Browse files
committed
Fix bogus "out of memory" reports in tuplestore.c.
The tuplesort/tuplestore memory management logic assumed that the chunkallocation overhead for its memtuples array could not increase whenincreasing the array size. This is and always was true for tuplesort,but we (I, I think) blindly copied that logic into tuplestore.c withoutnoticing that the assumption failed to hold for the much smaller arrayelements used by tuplestore. Given rather small work_mem, this couldresult in an improper complaint about "unexpected out-of-memory situation",as reported by Brent DeSpain in bug #13530.The easiest way to fix this is just to increase tuplestore's initialarray size so that the assumption holds. Rather than relying on magicconstants, though, let's export a #define from aset.c that representsthe safe allocation threshold, and make tuplestore's calculation dependon that.Do the same in tuplesort.c to keep the logic looking parallel, even thoughtuplesort.c isn't actually at risk at present. This will keep us frombreaking it if we ever muck with the allocation parameters in aset.c.Back-patch to all supported versions. The error message doesn't occurpre-9.3, not so much because the problem can't happen as because thepre-9.3 tuplestore code neglected to check for it. (The chance oftrouble is a great deal larger as of 9.3, though, due to changes in thearray-size-increasing strategy.) However, allowing LACKMEM() to becometrue unexpectedly could still result in less-than-desirable behavior,so let's patch it all the way back.
1 parentb58e8ca commit118c9bb

File tree

4 files changed

+42
-15
lines changed

4 files changed

+42
-15
lines changed

‎src/backend/utils/mmgr/aset.c

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -112,9 +112,9 @@
112112
*
113113
* With the current parameters, request sizes up to 8K are treated as chunks,
114114
* larger requests go into dedicated blocks. Change ALLOCSET_NUM_FREELISTS
115-
* to adjust the boundary point. (But in contexts with small maxBlockSize,
116-
*we may set the allocChunkLimit to less than 8K, so as to avoid space
117-
* wastage.)
115+
* to adjust the boundary point; and adjust ALLOCSET_SEPARATE_THRESHOLD in
116+
*memutils.h to agree. (Note: in contexts with small maxBlockSize, we may
117+
*set the allocChunkLimit to less than 8K, so as to avoid spacewastage.)
118118
*--------------------
119119
*/
120120

@@ -476,7 +476,12 @@ AllocSetContextCreate(MemoryContext parent,
476476
* We have to have allocChunkLimit a power of two, because the requested
477477
* and actually-allocated sizes of any chunk must be on the same side of
478478
* the limit, else we get confused about whether the chunk is "big".
479+
*
480+
* Also, allocChunkLimit must not exceed ALLOCSET_SEPARATE_THRESHOLD.
479481
*/
482+
StaticAssertStmt(ALLOC_CHUNK_LIMIT==ALLOCSET_SEPARATE_THRESHOLD,
483+
"ALLOC_CHUNK_LIMIT != ALLOCSET_SEPARATE_THRESHOLD");
484+
480485
context->allocChunkLimit=ALLOC_CHUNK_LIMIT;
481486
while ((Size) (context->allocChunkLimit+ALLOC_CHUNKHDRSZ)>
482487
(Size) ((maxBlockSize-ALLOC_BLOCKHDRSZ) /ALLOC_CHUNK_FRACTION))

‎src/backend/utils/sort/tuplesort.c

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -571,7 +571,14 @@ tuplesort_begin_common(int workMem, bool randomAccess)
571571
state->tapeset=NULL;
572572

573573
state->memtupcount=0;
574-
state->memtupsize=1024;/* initial guess */
574+
575+
/*
576+
* Initial size of array must be more than ALLOCSET_SEPARATE_THRESHOLD;
577+
* see comments in grow_memtuples().
578+
*/
579+
state->memtupsize=Max(1024,
580+
ALLOCSET_SEPARATE_THRESHOLD /sizeof(SortTuple)+1);
581+
575582
state->growmemtuples= true;
576583
state->memtuples= (SortTuple*)palloc(state->memtupsize*sizeof(SortTuple));
577584

@@ -1064,10 +1071,10 @@ grow_memtuples(Tuplesortstate *state)
10641071
* never generate a dangerous request, but to be safe, check explicitly
10651072
* that the array growth fits within availMem. (We could still cause
10661073
* LACKMEM if the memory chunk overhead associated with the memtuples
1067-
* array were to increase. That shouldn't happenwith any sane value of
1068-
*allowedMem, because at anyarray size large enough torisk LACKMEM,
1069-
*palloc would be treatingboth old and new arrays as separate chunks.
1070-
*But we'll check LACKMEMexplicitly below just in case.)
1074+
* array were to increase. That shouldn't happenbecause we chose the
1075+
*initialarray size large enough toensure that palloc will be treating
1076+
* both old and new arrays as separate chunks. But we'll check LACKMEM
1077+
* explicitly below just in case.)
10711078
*/
10721079
if (state->availMem< (int64) ((newmemtupsize-memtupsize)*sizeof(SortTuple)))
10731080
gotonoalloc;
@@ -1080,7 +1087,7 @@ grow_memtuples(Tuplesortstate *state)
10801087
state->memtupsize*sizeof(SortTuple));
10811088
USEMEM(state,GetMemoryChunkSpace(state->memtuples));
10821089
if (LACKMEM(state))
1083-
elog(ERROR,"unexpected out-of-memory situationduring sort");
1090+
elog(ERROR,"unexpected out-of-memory situationin tuplesort");
10841091
return true;
10851092

10861093
noalloc:

‎src/backend/utils/sort/tuplestore.c

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,14 @@ tuplestore_begin_common(int eflags, bool interXact, int maxKBytes)
265265

266266
state->memtupdeleted=0;
267267
state->memtupcount=0;
268-
state->memtupsize=1024;/* initial guess */
268+
269+
/*
270+
* Initial size of array must be more than ALLOCSET_SEPARATE_THRESHOLD;
271+
* see comments in grow_memtuples().
272+
*/
273+
state->memtupsize=Max(16384 /sizeof(void*),
274+
ALLOCSET_SEPARATE_THRESHOLD /sizeof(void*)+1);
275+
269276
state->growmemtuples= true;
270277
state->memtuples= (void**)palloc(state->memtupsize*sizeof(void*));
271278

@@ -639,10 +646,10 @@ grow_memtuples(Tuplestorestate *state)
639646
* never generate a dangerous request, but to be safe, check explicitly
640647
* that the array growth fits within availMem. (We could still cause
641648
* LACKMEM if the memory chunk overhead associated with the memtuples
642-
* array were to increase. That shouldn't happenwith any sane value of
643-
*allowedMem, because at anyarray size large enough torisk LACKMEM,
644-
*palloc would be treatingboth old and new arrays as separate chunks.
645-
*But we'll check LACKMEMexplicitly below just in case.)
649+
* array were to increase. That shouldn't happenbecause we chose the
650+
*initialarray size large enough toensure that palloc will be treating
651+
* both old and new arrays as separate chunks. But we'll check LACKMEM
652+
* explicitly below just in case.)
646653
*/
647654
if (state->availMem< (int64) ((newmemtupsize-memtupsize)*sizeof(void*)))
648655
gotonoalloc;
@@ -655,7 +662,7 @@ grow_memtuples(Tuplestorestate *state)
655662
state->memtupsize*sizeof(void*));
656663
USEMEM(state,GetMemoryChunkSpace(state->memtuples));
657664
if (LACKMEM(state))
658-
elog(ERROR,"unexpected out-of-memory situationduring sort");
665+
elog(ERROR,"unexpected out-of-memory situationin tuplestore");
659666
return true;
660667

661668
noalloc:

‎src/include/utils/memutils.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,4 +145,12 @@ extern MemoryContext AllocSetContextCreate(MemoryContext parent,
145145
#defineALLOCSET_SMALL_INITSIZE (1 * 1024)
146146
#defineALLOCSET_SMALL_MAXSIZE (8 * 1024)
147147

148+
/*
149+
* Threshold above which a request in an AllocSet context is certain to be
150+
* allocated separately (and thereby have constant allocation overhead).
151+
* Few callers should be interested in this, but tuplesort/tuplestore need
152+
* to know it.
153+
*/
154+
#defineALLOCSET_SEPARATE_THRESHOLD 8192
155+
148156
#endif/* MEMUTILS_H */

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp