Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitf7672c8

Browse files
committed
Avoid buffer bloat in libpq when server is consistently faster than client.
If the server sends a long stream of data, and the server + network areconsistently fast enough to force the recv() loop in pqReadData() toiterate until libpq's input buffer is full, then upon processing the lastincomplete message in each bufferload we'd usually double the buffer size,due to supposing that we didn't have enough room in the buffer to finishcollecting that message. After filling the newly-enlarged buffer, thecycle repeats, eventually resulting in an out-of-memory situation (whichwould be reported misleadingly as "lost synchronization with server").Of course, we should not enlarge the buffer unless we still need roomafter discarding already-processed messages.This bug dates back quite a long time: pqParseInput3 has had the behaviorsince perhaps 2003, getCopyDataMessage at least since commit70066ebin 2008. Probably the reason it's not been isolated before is that incommon environments the recv() loop would always be faster than the server(if on the same machine) or faster than the network (if not); or at leastit wouldn't be slower consistently enough to let the buffer ramp up to aproblematic size. The reported cases involve Windows, which perhaps hasdifferent timing behavior than other platforms.Per bug #7914 from Shin-ichi Morita, though this is different from hisproposed solution. Back-patch to all supported branches.
1 parent022b5f2 commitf7672c8

File tree

1 file changed

+32
-0
lines changed

1 file changed

+32
-0
lines changed

‎src/interfaces/libpq/fe-misc.c

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -352,6 +352,7 @@ pqCheckOutBufferSpace(size_t bytes_needed, PGconn *conn)
352352
intnewsize=conn->outBufSize;
353353
char*newbuf;
354354

355+
/* Quick exit if we have enough space */
355356
if (bytes_needed <= (size_t)newsize)
356357
return0;
357358

@@ -415,6 +416,37 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
415416
intnewsize=conn->inBufSize;
416417
char*newbuf;
417418

419+
/* Quick exit if we have enough space */
420+
if (bytes_needed <= (size_t)newsize)
421+
return0;
422+
423+
/*
424+
* Before concluding that we need to enlarge the buffer, left-justify
425+
* whatever is in it and recheck. The caller's value of bytes_needed
426+
* includes any data to the left of inStart, but we can delete that in
427+
* preference to enlarging the buffer. It's slightly ugly to have this
428+
* function do this, but it's better than making callers worry about it.
429+
*/
430+
bytes_needed-=conn->inStart;
431+
432+
if (conn->inStart<conn->inEnd)
433+
{
434+
if (conn->inStart>0)
435+
{
436+
memmove(conn->inBuffer,conn->inBuffer+conn->inStart,
437+
conn->inEnd-conn->inStart);
438+
conn->inEnd-=conn->inStart;
439+
conn->inCursor-=conn->inStart;
440+
conn->inStart=0;
441+
}
442+
}
443+
else
444+
{
445+
/* buffer is logically empty, reset it */
446+
conn->inStart=conn->inCursor=conn->inEnd=0;
447+
}
448+
449+
/* Recheck whether we have enough space */
418450
if (bytes_needed <= (size_t)newsize)
419451
return0;
420452

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp