Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitdbbae33

Browse files
committed
Delay fsyncs of pg_basebackup until the end of backup
Since the addition of fsync requests in bc34223 to make base backup dataconsistent on disk once pg_basebackup finishes, each tablespace tar fileis individually flushed once completed, with an additional flush of theparent directory when the base backup finishes. While holding aconnection to the server, a fsync request taking a long time may cause afailure of the base backup, which is annoying for any integration. Arecent example of breakage can involve tcp_user_timeout, butwal_sender_timeout can cause similar problems.While reviewing the code, there was a second issue causing too manyfsync requests to be done for the same WAL data. As recursive fsyncsare done at the end of the backup for both the plain and tar formatsfrom the base target directory where everything is written, it is fineto disable fsyncs when fetching or streaming WAL.Reported-by: Ryohei TakahashiAuthor: Michael PaquierReviewed-by: Ryohei TakahashiDiscussion:https://postgr.es/m/OSBPR01MB4550DAE2F8C9502894A45AAB82BE0@OSBPR01MB4550.jpnprd01.prod.outlook.comBackpatch-through: 10
1 parent604d206 commitdbbae33

File tree

1 file changed

+14
-10
lines changed

1 file changed

+14
-10
lines changed

‎src/bin/pg_basebackup/pg_basebackup.c

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -492,7 +492,8 @@ LogStreamerMain(logstreamer_param *param)
492492
#endif
493493
stream.standby_message_timeout=standby_message_timeout;
494494
stream.synchronous= false;
495-
stream.do_sync=do_sync;
495+
/* fsync happens at the end of pg_basebackup for all data */
496+
stream.do_sync= false;
496497
stream.mark_done= true;
497498
stream.partial_suffix=NULL;
498499
stream.replication_slot=replication_slot;
@@ -501,9 +502,11 @@ LogStreamerMain(logstreamer_param *param)
501502
stream.replication_slot=psprintf("pg_basebackup_%d", (int)PQbackendPID(param->bgconn));
502503

503504
if (format=='p')
504-
stream.walmethod=CreateWalDirectoryMethod(param->xlog,0,do_sync);
505+
stream.walmethod=CreateWalDirectoryMethod(param->xlog,0,
506+
stream.do_sync);
505507
else
506-
stream.walmethod=CreateWalTarMethod(param->xlog,compresslevel,do_sync);
508+
stream.walmethod=CreateWalTarMethod(param->xlog,compresslevel,
509+
stream.do_sync);
507510

508511
if (!ReceiveXlogStream(param->bgconn,&stream))
509512

@@ -1274,9 +1277,10 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
12741277
if (copybuf!=NULL)
12751278
PQfreemem(copybuf);
12761279

1277-
/* sync the resulting tar file, errors are not considered fatal */
1278-
if (do_sync&&strcmp(basedir,"-")!=0)
1279-
(void)fsync_fname(filename, false,progname);
1280+
/*
1281+
* Do not sync the resulting tar file yet, all files are synced once at
1282+
* the end.
1283+
*/
12801284
}
12811285

12821286

@@ -2056,17 +2060,17 @@ BaseBackup(void)
20562060

20572061
/*
20582062
* Make data persistent on disk once backup is completed. For tar format
2059-
*once syncingthe parent directoryis fine,each tar filecreated per
2060-
*tablespace has been already synced.In plain format, all the data of
2061-
*thebase directory is synced, taking into account all the tablespaces.
2063+
*syncthe parent directoryand all its contents aseach tar filewas not
2064+
*synced after being completed.In plain format, all the data of the
2065+
* base directory is synced, taking into account all the tablespaces.
20622066
* Errors are not considered fatal.
20632067
*/
20642068
if (do_sync)
20652069
{
20662070
if (format=='t')
20672071
{
20682072
if (strcmp(basedir,"-")!=0)
2069-
(void)fsync_fname(basedir, true,progname);
2073+
(void)fsync_dir_recurse(basedir,progname);
20702074
}
20712075
else
20722076
{

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp