Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit8ea3e7a

Browse files
committed
Fix bogus "out of memory" reports in tuplestore.c.
The tuplesort/tuplestore memory management logic assumed that the chunkallocation overhead for its memtuples array could not increase whenincreasing the array size. This is and always was true for tuplesort,but we (I, I think) blindly copied that logic into tuplestore.c withoutnoticing that the assumption failed to hold for the much smaller arrayelements used by tuplestore. Given rather small work_mem, this couldresult in an improper complaint about "unexpected out-of-memory situation",as reported by Brent DeSpain in bug #13530.The easiest way to fix this is just to increase tuplestore's initialarray size so that the assumption holds. Rather than relying on magicconstants, though, let's export a #define from aset.c that representsthe safe allocation threshold, and make tuplestore's calculation dependon that.Do the same in tuplesort.c to keep the logic looking parallel, even thoughtuplesort.c isn't actually at risk at present. This will keep us frombreaking it if we ever muck with the allocation parameters in aset.c.Back-patch to all supported versions. The error message doesn't occurpre-9.3, not so much because the problem can't happen as because thepre-9.3 tuplestore code neglected to check for it. (The chance oftrouble is a great deal larger as of 9.3, though, due to changes in thearray-size-increasing strategy.) However, allowing LACKMEM() to becometrue unexpectedly could still result in less-than-desirable behavior,so let's patch it all the way back.
1 parent85e5e22 commit8ea3e7a

File tree

4 files changed

+42
-15
lines changed

4 files changed

+42
-15
lines changed

‎src/backend/utils/mmgr/aset.c

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -112,9 +112,9 @@
112112
*
113113
* With the current parameters, request sizes up to 8K are treated as chunks,
114114
* larger requests go into dedicated blocks. Change ALLOCSET_NUM_FREELISTS
115-
* to adjust the boundary point. (But in contexts with small maxBlockSize,
116-
*we may set the allocChunkLimit to less than 8K, so as to avoid space
117-
* wastage.)
115+
* to adjust the boundary point; and adjust ALLOCSET_SEPARATE_THRESHOLD in
116+
*memutils.h to agree. (Note: in contexts with small maxBlockSize, we may
117+
*set the allocChunkLimit to less than 8K, so as to avoid spacewastage.)
118118
*--------------------
119119
*/
120120

@@ -476,7 +476,12 @@ AllocSetContextCreate(MemoryContext parent,
476476
* We have to have allocChunkLimit a power of two, because the requested
477477
* and actually-allocated sizes of any chunk must be on the same side of
478478
* the limit, else we get confused about whether the chunk is "big".
479+
*
480+
* Also, allocChunkLimit must not exceed ALLOCSET_SEPARATE_THRESHOLD.
479481
*/
482+
StaticAssertStmt(ALLOC_CHUNK_LIMIT==ALLOCSET_SEPARATE_THRESHOLD,
483+
"ALLOC_CHUNK_LIMIT != ALLOCSET_SEPARATE_THRESHOLD");
484+
480485
set->allocChunkLimit=ALLOC_CHUNK_LIMIT;
481486
while ((Size) (set->allocChunkLimit+ALLOC_CHUNKHDRSZ)>
482487
(Size) ((maxBlockSize-ALLOC_BLOCKHDRSZ) /ALLOC_CHUNK_FRACTION))

‎src/backend/utils/sort/tuplesort.c

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -577,7 +577,14 @@ tuplesort_begin_common(int workMem, bool randomAccess)
577577
state->tapeset=NULL;
578578

579579
state->memtupcount=0;
580-
state->memtupsize=1024;/* initial guess */
580+
581+
/*
582+
* Initial size of array must be more than ALLOCSET_SEPARATE_THRESHOLD;
583+
* see comments in grow_memtuples().
584+
*/
585+
state->memtupsize=Max(1024,
586+
ALLOCSET_SEPARATE_THRESHOLD /sizeof(SortTuple)+1);
587+
581588
state->growmemtuples= true;
582589
state->memtuples= (SortTuple*)palloc(state->memtupsize*sizeof(SortTuple));
583590

@@ -1165,10 +1172,10 @@ grow_memtuples(Tuplesortstate *state)
11651172
* never generate a dangerous request, but to be safe, check explicitly
11661173
* that the array growth fits within availMem. (We could still cause
11671174
* LACKMEM if the memory chunk overhead associated with the memtuples
1168-
* array were to increase. That shouldn't happenwith any sane value of
1169-
*allowedMem, because at anyarray size large enough torisk LACKMEM,
1170-
*palloc would be treatingboth old and new arrays as separate chunks.
1171-
*But we'll check LACKMEMexplicitly below just in case.)
1175+
* array were to increase. That shouldn't happenbecause we chose the
1176+
*initialarray size large enough toensure that palloc will be treating
1177+
* both old and new arrays as separate chunks. But we'll check LACKMEM
1178+
* explicitly below just in case.)
11721179
*/
11731180
if (state->availMem< (int64) ((newmemtupsize-memtupsize)*sizeof(SortTuple)))
11741181
gotonoalloc;
@@ -1181,7 +1188,7 @@ grow_memtuples(Tuplesortstate *state)
11811188
state->memtupsize*sizeof(SortTuple));
11821189
USEMEM(state,GetMemoryChunkSpace(state->memtuples));
11831190
if (LACKMEM(state))
1184-
elog(ERROR,"unexpected out-of-memory situationduring sort");
1191+
elog(ERROR,"unexpected out-of-memory situationin tuplesort");
11851192
return true;
11861193

11871194
noalloc:

‎src/backend/utils/sort/tuplestore.c

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -267,7 +267,14 @@ tuplestore_begin_common(int eflags, bool interXact, int maxKBytes)
267267

268268
state->memtupdeleted=0;
269269
state->memtupcount=0;
270-
state->memtupsize=1024;/* initial guess */
270+
271+
/*
272+
* Initial size of array must be more than ALLOCSET_SEPARATE_THRESHOLD;
273+
* see comments in grow_memtuples().
274+
*/
275+
state->memtupsize=Max(16384 /sizeof(void*),
276+
ALLOCSET_SEPARATE_THRESHOLD /sizeof(void*)+1);
277+
271278
state->growmemtuples= true;
272279
state->memtuples= (void**)palloc(state->memtupsize*sizeof(void*));
273280

@@ -641,10 +648,10 @@ grow_memtuples(Tuplestorestate *state)
641648
* never generate a dangerous request, but to be safe, check explicitly
642649
* that the array growth fits within availMem. (We could still cause
643650
* LACKMEM if the memory chunk overhead associated with the memtuples
644-
* array were to increase. That shouldn't happenwith any sane value of
645-
*allowedMem, because at anyarray size large enough torisk LACKMEM,
646-
*palloc would be treatingboth old and new arrays as separate chunks.
647-
*But we'll check LACKMEMexplicitly below just in case.)
651+
* array were to increase. That shouldn't happenbecause we chose the
652+
*initialarray size large enough toensure that palloc will be treating
653+
* both old and new arrays as separate chunks. But we'll check LACKMEM
654+
* explicitly below just in case.)
648655
*/
649656
if (state->availMem< (int64) ((newmemtupsize-memtupsize)*sizeof(void*)))
650657
gotonoalloc;
@@ -657,7 +664,7 @@ grow_memtuples(Tuplestorestate *state)
657664
state->memtupsize*sizeof(void*));
658665
USEMEM(state,GetMemoryChunkSpace(state->memtuples));
659666
if (LACKMEM(state))
660-
elog(ERROR,"unexpected out-of-memory situationduring sort");
667+
elog(ERROR,"unexpected out-of-memory situationin tuplestore");
661668
return true;
662669

663670
noalloc:

‎src/include/utils/memutils.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,4 +150,12 @@ extern MemoryContext AllocSetContextCreate(MemoryContext parent,
150150
#defineALLOCSET_SMALL_INITSIZE (1 * 1024)
151151
#defineALLOCSET_SMALL_MAXSIZE (8 * 1024)
152152

153+
/*
154+
* Threshold above which a request in an AllocSet context is certain to be
155+
* allocated separately (and thereby have constant allocation overhead).
156+
* Few callers should be interested in this, but tuplesort/tuplestore need
157+
* to know it.
158+
*/
159+
#defineALLOCSET_SEPARATE_THRESHOLD 8192
160+
153161
#endif/* MEMUTILS_H */

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp