Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit16d0da5

Browse files
committed
Don't try to read a multi-GB pg_stat_statements file in one call.
Windows fails on a request to read() more than INT_MAX bytes,and perhaps other platforms could have similar issues. Let'sadjust this code to read at most 1GB per call.(One would not have thought the file could get that big, but nowwe have a field report of trouble, so it can. We likely ought toadd some mechanism to limit the size of the query-texts fileseparately from the size of the hash table. That is not thispatch, though.)Per bug #17254 from Yusuke Egashira. It's been like this forawhile, so back-patch to all supported branches.Discussion:https://postgr.es/m/17254-a926c89dc03375c2@postgresql.org
1 parent14b8d25 commit16d0da5

File tree

1 file changed

+29
-16
lines changed

1 file changed

+29
-16
lines changed

‎contrib/pg_stat_statements/pg_stat_statements.c

Lines changed: 29 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1933,6 +1933,7 @@ qtext_load_file(Size *buffer_size)
19331933
char*buf;
19341934
intfd;
19351935
structstatstat;
1936+
Sizenread;
19361937

19371938
fd=OpenTransientFile(PGSS_TEXT_FILE,O_RDONLY |PG_BINARY);
19381939
if (fd<0)
@@ -1973,31 +1974,43 @@ qtext_load_file(Size *buffer_size)
19731974
}
19741975

19751976
/*
1976-
* OK, slurp in the file. If we get a short read and errno doesn't get
1977-
* set, the reason is probably that garbage collection truncated the file
1978-
* since we did the fstat(), so we don't log a complaint --- but we don't
1979-
* return the data, either, since it's most likely corrupt due to
1980-
* concurrent writes from garbage collection.
1977+
* OK, slurp in the file. Windows fails if we try to read more than
1978+
* INT_MAX bytes at once, and other platforms might not like that either,
1979+
* so read a very large file in 1GB segments.
19811980
*/
1982-
errno=0;
1983-
if (read(fd,buf,stat.st_size)!=stat.st_size)
1981+
nread=0;
1982+
while (nread<stat.st_size)
19841983
{
1985-
if (errno)
1986-
ereport(LOG,
1987-
(errcode_for_file_access(),
1988-
errmsg("could not read file \"%s\": %m",
1989-
PGSS_TEXT_FILE)));
1990-
free(buf);
1991-
CloseTransientFile(fd);
1992-
returnNULL;
1984+
inttoread=Min(1024*1024*1024,stat.st_size-nread);
1985+
1986+
/*
1987+
* If we get a short read and errno doesn't get set, the reason is
1988+
* probably that garbage collection truncated the file since we did
1989+
* the fstat(), so we don't log a complaint --- but we don't return
1990+
* the data, either, since it's most likely corrupt due to concurrent
1991+
* writes from garbage collection.
1992+
*/
1993+
errno=0;
1994+
if (read(fd,buf+nread,toread)!=toread)
1995+
{
1996+
if (errno)
1997+
ereport(LOG,
1998+
(errcode_for_file_access(),
1999+
errmsg("could not read file \"%s\": %m",
2000+
PGSS_TEXT_FILE)));
2001+
free(buf);
2002+
CloseTransientFile(fd);
2003+
returnNULL;
2004+
}
2005+
nread+=toread;
19932006
}
19942007

19952008
if (CloseTransientFile(fd))
19962009
ereport(LOG,
19972010
(errcode_for_file_access(),
19982011
errmsg("could not close file \"%s\": %m",PGSS_TEXT_FILE)));
19992012

2000-
*buffer_size=stat.st_size;
2013+
*buffer_size=nread;
20012014
returnbuf;
20022015
}
20032016

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp