Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit33766e6

Browse files
committed
Here's a patch against 7.1.3 that fixes a problem with sending larger
queries over non-blocking connections with libpq. "Larger" herebasically means that it doesn't fit into the output buffer.The basic strategy is to fix pqFlush and pqPutBytes.The problem with pqFlush as it stands now is that it returns EOF when anerror occurs or when not all data could be sent. The latter case isclearly not an error for a non-blocking connection but the caller can'tdistringuish it from an error very well.The first part of the fix is therefore to fix pqFlush. This is done byto renaming it to pqSendSome which only differs from pqFlush in itsreturn values to allow the caller to make the above distinction and anew pqFlush which is implemented in terms of pqSendSome and behavesexactly like the old pqFlush.The second part of the fix modifies pqPutBytes to use pqSendSome insteadof pqFlush and to either send all the data or if not all data can besent on a non-blocking connection to at least put all data into theoutput buffer, enlarging it if necessary. The callers of pqPutBytesdon't have to be changed because from their point of view pqPutBytesbehaves like before. It either succeeds in queueing all output data orfails with an error.I've also added a new API function PQsendSome which analogously toPQflush just calls pqSendSome. Programs using non-blocking queriesshould use this new function. The main difference is that this functionwill have to be called repeatedly (calling select() properly in between)until all data has been written.AFAICT, the code in CVS HEAD hasn't changed with respect to non-blockingqueries and this fix should work there, too, but I haven't tested thatyet.Bernhard Herzog
1 parent9446718 commit33766e6

File tree

4 files changed

+128
-57
lines changed

4 files changed

+128
-57
lines changed

‎src/interfaces/libpq/fe-exec.c

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
*
99
*
1010
* IDENTIFICATION
11-
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v 1.114 2002/03/04 23:59:14 momjian Exp $
11+
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v 1.115 2002/03/05 05:20:12 momjian Exp $
1212
*
1313
*-------------------------------------------------------------------------
1414
*/
@@ -2429,3 +2429,11 @@ PQflush(PGconn *conn)
24292429

24302430
return (pqFlush(conn));
24312431
}
2432+
2433+
/* try to force data out, really only useful for non-blocking users.
2434+
* This implementation actually works for non-blocking connections */
2435+
int
2436+
PQsendSome(PGconn*conn)
2437+
{
2438+
returnpqSendSome(conn);
2439+
}

‎src/interfaces/libpq/fe-misc.c

Lines changed: 111 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
*
2626
*
2727
* IDENTIFICATION
28-
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-misc.c,v 1.65 2001/12/03 00:28:24 tgl Exp $
28+
* $Header: /cvsroot/pgsql/src/interfaces/libpq/fe-misc.c,v 1.66 2002/03/05 05:20:12 momjian Exp $
2929
*
3030
*-------------------------------------------------------------------------
3131
*/
@@ -110,54 +110,82 @@ pqPutc(char c, PGconn *conn)
110110
staticint
111111
pqPutBytes(constchar*s,size_tnbytes,PGconn*conn)
112112
{
113-
size_tavail=Max(conn->outBufSize-conn->outCount,0);
114-
115113
/*
116-
* if we are non-blocking and the send queue is too full to buffer
117-
* this request then try to flush some and return an error
114+
* Strategy to handle blocking and non-blocking connections: Fill the
115+
* output buffer and flush it repeatedly until either all data has
116+
* been sent or is at least queued in the buffer.
117+
*
118+
* For non-blocking connections, grow the buffer if not all data fits
119+
* into it and the buffer can't be sent because the socket would
120+
* block.
118121
*/
119-
if (pqIsnonblocking(conn)&&nbytes>avail&&pqFlush(conn))
122+
123+
while (nbytes)
120124
{
125+
size_tavail,
126+
remaining;
127+
128+
/* fill the output buffer */
129+
avail=Max(conn->outBufSize-conn->outCount,0);
130+
remaining=Min(avail,nbytes);
131+
memcpy(conn->outBuffer+conn->outCount,s,remaining);
132+
conn->outCount+=remaining;
133+
s+=remaining;
134+
nbytes-=remaining;
135+
121136
/*
122-
* even if the flush failed we may still have written some data,
123-
* recalculate the size of the send-queue relative to the amount
124-
* we have to send, we may be able to queue it afterall even
125-
* though it's not sent to the database it's ok, any routines that
126-
* check the data coming from the database better call pqFlush()
127-
* anyway.
137+
* if the data didn't fit completely into the buffer, try to flush
138+
* the buffer
128139
*/
129-
if (nbytes>Max(conn->outBufSize-conn->outCount,0))
140+
if (nbytes)
130141
{
131-
printfPQExpBuffer(&conn->errorMessage,
132-
libpq_gettext("could not flush enough data (space available: %d, space needed %d)\n"),
133-
(int)Max(conn->outBufSize-conn->outCount,0),
134-
(int)nbytes);
135-
returnEOF;
136-
}
137-
/* fixup avail for while loop */
138-
avail=Max(conn->outBufSize-conn->outCount,0);
139-
}
142+
intsend_result=pqSendSome(conn);
140143

141-
/*
142-
* is the amount of data to be sent is larger than the size of the
143-
* output buffer then we must flush it to make more room.
144-
*
145-
* the code above will make sure the loop conditional is never true for
146-
* non-blocking connections
147-
*/
148-
while (nbytes>avail)
149-
{
150-
memcpy(conn->outBuffer+conn->outCount,s,avail);
151-
conn->outCount+=avail;
152-
s+=avail;
153-
nbytes-=avail;
154-
if (pqFlush(conn))
155-
returnEOF;
156-
avail=conn->outBufSize;
157-
}
144+
/* if there were errors, report them */
145+
if (send_result<0)
146+
returnEOF;
147+
148+
/*
149+
* if not all data could be sent, increase the output buffer,
150+
* put the rest of s into it and return successfully. This
151+
* case will only happen in a non-blocking connection
152+
*/
153+
if (send_result>0)
154+
{
155+
/*
156+
* try to grow the buffer. FIXME: The new size could be
157+
* chosen more intelligently.
158+
*/
159+
size_tbuflen=conn->outCount+nbytes;
160+
161+
if (buflen>conn->outBufSize)
162+
{
163+
char*newbuf=realloc(conn->outBuffer,buflen);
164+
165+
if (!newbuf)
166+
{
167+
/* realloc failed. Probably out of memory */
168+
printfPQExpBuffer(&conn->errorMessage,
169+
"cannot allocate memory for output buffer\n");
170+
returnEOF;
171+
}
172+
conn->outBuffer=newbuf;
173+
conn->outBufSize=buflen;
174+
}
175+
/* put the data into it */
176+
memcpy(conn->outBuffer+conn->outCount,s,nbytes);
177+
conn->outCount+=nbytes;
178+
179+
/* report success. */
180+
return0;
181+
}
182+
}
158183

159-
memcpy(conn->outBuffer+conn->outCount,s,nbytes);
160-
conn->outCount+=nbytes;
184+
/*
185+
* pqSendSome was able to send all data. Continue with the next
186+
* chunk of s.
187+
*/
188+
}/* while */
161189

162190
return0;
163191
}
@@ -603,11 +631,22 @@ pqReadData(PGconn *conn)
603631
return-1;
604632
}
605633

606-
/*
607-
* pqFlush: send any data waiting in the output buffer
634+
/* pqSendSome: send any data waiting in the output buffer and return 0
635+
* if all data was sent, -1 if an error occurred or 1 if not all data
636+
* could be written because the socket would have blocked.
637+
*
638+
* For a blocking connection all data will be sent unless an error
639+
* occurrs. -1 will only be returned if the connection is non-blocking.
640+
*
641+
* Internally, the case of data remaining in the buffer after pqSendSome
642+
* could be determined by looking at outCount, but this function also
643+
* serves as the implementation of the new API function PQsendsome.
644+
*
645+
* FIXME: perhaps it would be more useful to return the number of bytes
646+
* remaining?
608647
*/
609648
int
610-
pqFlush(PGconn*conn)
649+
pqSendSome(PGconn*conn)
611650
{
612651
char*ptr=conn->outBuffer;
613652
intlen=conn->outCount;
@@ -685,14 +724,14 @@ pqFlush(PGconn *conn)
685724
* the socket open until pqReadData finds no more data
686725
* can be read.
687726
*/
688-
returnEOF;
727+
return-1;
689728

690729
default:
691730
printfPQExpBuffer(&conn->errorMessage,
692731
libpq_gettext("could not send data to server: %s\n"),
693732
SOCK_STRERROR(SOCK_ERRNO));
694733
/* We don't assume it's a fatal error... */
695-
returnEOF;
734+
return-1;
696735
}
697736
}
698737
else
@@ -707,7 +746,7 @@ pqFlush(PGconn *conn)
707746

708747
/*
709748
* if the socket is in non-blocking mode we may need to abort
710-
* here
749+
* here and return 1 to indicate that data is still pending.
711750
*/
712751
#ifdefUSE_SSL
713752
/* can't do anything for our SSL users yet */
@@ -719,14 +758,14 @@ pqFlush(PGconn *conn)
719758
/* shift the contents of the buffer */
720759
memmove(conn->outBuffer,ptr,len);
721760
conn->outCount=len;
722-
returnEOF;
761+
return1;
723762
}
724763
#ifdefUSE_SSL
725764
}
726765
#endif
727766

728767
if (pqWait(FALSE, TRUE,conn))
729-
returnEOF;
768+
return-1;
730769
}
731770
}
732771

@@ -738,6 +777,23 @@ pqFlush(PGconn *conn)
738777
return0;
739778
}
740779

780+
781+
/*
782+
* pqFlush: send any data waiting in the output buffer
783+
*
784+
* Implemented in terms of pqSendSome to recreate the old behavior which
785+
* returned 0 if all data was sent or EOF. EOF was sent regardless of
786+
* whether an error occurred or not all data was sent on a non-blocking
787+
* socket.
788+
*/
789+
int
790+
pqFlush(PGconn*conn)
791+
{
792+
if (pqSendSome(conn))
793+
returnEOF;
794+
return0;
795+
}
796+
741797
/*
742798
* pqWait: wait until we can read or write the connection socket
743799
*
@@ -756,7 +812,7 @@ pqWait(int forRead, int forWrite, PGconn *conn)
756812
{
757813
printfPQExpBuffer(&conn->errorMessage,
758814
libpq_gettext("connection not open\n"));
759-
returnEOF;
815+
return-1;
760816
}
761817

762818
if (forRead||forWrite)
@@ -857,19 +913,20 @@ libpq_gettext(const char *msgid)
857913
* strerror replacement for windows:
858914
*
859915
* This works on WIN2000 and newer, but we don't know where to find WinSock
860-
* error strings on older Windows flavors.If you know, clue us in.
916+
* error strings on older Windows flavors.If you know, clue us in.
861917
*/
862918
constchar*
863919
winsock_strerror(inteno)
864920
{
865-
staticcharerr_buf[512];
866-
#defineWSSE_MAXLEN (sizeof(err_buf)-1-13)/* 13 for " (0x00000000)" */
921+
staticcharerr_buf[512];
922+
923+
#defineWSSE_MAXLEN (sizeof(err_buf)-1-13)/* 13 for " (0x00000000)" */
867924
intlength;
868925

869926
/* First try the "system table", this works on Win2k and up */
870927

871928
if (FormatMessage(
872-
FORMAT_MESSAGE_IGNORE_INSERTS |FORMAT_MESSAGE_FROM_SYSTEM,
929+
FORMAT_MESSAGE_IGNORE_INSERTS |FORMAT_MESSAGE_FROM_SYSTEM,
873930
0,
874931
eno,
875932
MAKELANGID(LANG_NEUTRAL,SUBLANG_DEFAULT),

‎src/interfaces/libpq/libpq-fe.h

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
* Portions Copyright (c) 1996-2001, PostgreSQL Global Development Group
88
* Portions Copyright (c) 1994, Regents of the University of California
99
*
10-
* $Id: libpq-fe.h,v 1.81 2002/03/04 23:59:14 momjian Exp $
10+
* $Id: libpq-fe.h,v 1.82 2002/03/05 05:20:12 momjian Exp $
1111
*
1212
*-------------------------------------------------------------------------
1313
*/
@@ -282,6 +282,11 @@ extern intPQisnonblocking(const PGconn *conn);
282282

283283
/* Force the write buffer to be written (or at least try) */
284284
externintPQflush(PGconn*conn);
285+
/*
286+
* Force the write buffer to be written (or at least try)
287+
* (better than PQflush)
288+
*/
289+
externintPQsendSome(PGconn*conn);
285290

286291
/*
287292
* "Fast path" interface --- not really recommended for application

‎src/interfaces/libpq/libpq-int.h

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
* Portions Copyright (c) 1996-2001, PostgreSQL Global Development Group
1313
* Portions Copyright (c) 1994, Regents of the University of California
1414
*
15-
* $Id: libpq-int.h,v 1.44 2001/11/0517:46:38 momjian Exp $
15+
* $Id: libpq-int.h,v 1.45 2002/03/0505:20:12 momjian Exp $
1616
*
1717
*-------------------------------------------------------------------------
1818
*/
@@ -323,6 +323,7 @@ extern intpqGetInt(int *result, size_t bytes, PGconn *conn);
323323
externintpqPutInt(intvalue,size_tbytes,PGconn*conn);
324324
externintpqReadData(PGconn*conn);
325325
externintpqFlush(PGconn*conn);
326+
externintpqSendSome(PGconn*conn);
326327
externintpqWait(intforRead,intforWrite,PGconn*conn);
327328
externintpqReadReady(PGconn*conn);
328329
externintpqWriteReady(PGconn*conn);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp