Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit664ac3d

Browse files
committed
Avoid buffer bloat in libpq when server is consistently faster than client.
If the server sends a long stream of data, and the server + network areconsistently fast enough to force the recv() loop in pqReadData() toiterate until libpq's input buffer is full, then upon processing the lastincomplete message in each bufferload we'd usually double the buffer size,due to supposing that we didn't have enough room in the buffer to finishcollecting that message. After filling the newly-enlarged buffer, thecycle repeats, eventually resulting in an out-of-memory situation (whichwould be reported misleadingly as "lost synchronization with server").Of course, we should not enlarge the buffer unless we still need roomafter discarding already-processed messages.This bug dates back quite a long time: pqParseInput3 has had the behaviorsince perhaps 2003, getCopyDataMessage at least since commit70066ebin 2008. Probably the reason it's not been isolated before is that incommon environments the recv() loop would always be faster than the server(if on the same machine) or faster than the network (if not); or at leastit wouldn't be slower consistently enough to let the buffer ramp up to aproblematic size. The reported cases involve Windows, which perhaps hasdifferent timing behavior than other platforms.Per bug #7914 from Shin-ichi Morita, though this is different from hisproposed solution. Back-patch to all supported branches.
1 parent2a527ba commit664ac3d

File tree

1 file changed

+32
-0
lines changed

1 file changed

+32
-0
lines changed

‎src/interfaces/libpq/fe-misc.c

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -318,6 +318,7 @@ pqCheckOutBufferSpace(size_t bytes_needed, PGconn *conn)
318318
intnewsize=conn->outBufSize;
319319
char*newbuf;
320320

321+
/* Quick exit if we have enough space */
321322
if (bytes_needed <= (size_t)newsize)
322323
return0;
323324

@@ -381,6 +382,37 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
381382
intnewsize=conn->inBufSize;
382383
char*newbuf;
383384

385+
/* Quick exit if we have enough space */
386+
if (bytes_needed <= (size_t)newsize)
387+
return0;
388+
389+
/*
390+
* Before concluding that we need to enlarge the buffer, left-justify
391+
* whatever is in it and recheck. The caller's value of bytes_needed
392+
* includes any data to the left of inStart, but we can delete that in
393+
* preference to enlarging the buffer. It's slightly ugly to have this
394+
* function do this, but it's better than making callers worry about it.
395+
*/
396+
bytes_needed-=conn->inStart;
397+
398+
if (conn->inStart<conn->inEnd)
399+
{
400+
if (conn->inStart>0)
401+
{
402+
memmove(conn->inBuffer,conn->inBuffer+conn->inStart,
403+
conn->inEnd-conn->inStart);
404+
conn->inEnd-=conn->inStart;
405+
conn->inCursor-=conn->inStart;
406+
conn->inStart=0;
407+
}
408+
}
409+
else
410+
{
411+
/* buffer is logically empty, reset it */
412+
conn->inStart=conn->inCursor=conn->inEnd=0;
413+
}
414+
415+
/* Recheck whether we have enough space */
384416
if (bytes_needed <= (size_t)newsize)
385417
return0;
386418

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp