Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit522baf1

Browse files
committed
Delay fsyncs of pg_basebackup until the end of backup
Since the addition of fsync requests in bc34223 to make base backup dataconsistent on disk once pg_basebackup finishes, each tablespace tar fileis individually flushed once completed, with an additional flush of theparent directory when the base backup finishes. While holding aconnection to the server, a fsync request taking a long time may cause afailure of the base backup, which is annoying for any integration. Arecent example of breakage can involve tcp_user_timeout, butwal_sender_timeout can cause similar problems.While reviewing the code, there was a second issue causing too manyfsync requests to be done for the same WAL data. As recursive fsyncsare done at the end of the backup for both the plain and tar formatsfrom the base target directory where everything is written, it is fineto disable fsyncs when fetching or streaming WAL.Reported-by: Ryohei TakahashiAuthor: Michael PaquierReviewed-by: Ryohei TakahashiDiscussion:https://postgr.es/m/OSBPR01MB4550DAE2F8C9502894A45AAB82BE0@OSBPR01MB4550.jpnprd01.prod.outlook.comBackpatch-through: 10
1 parent25dcc9d commit522baf1

File tree

1 file changed

+14
-10
lines changed

1 file changed

+14
-10
lines changed

‎src/bin/pg_basebackup/pg_basebackup.c

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -486,15 +486,18 @@ LogStreamerMain(logstreamer_param *param)
486486
#endif
487487
stream.standby_message_timeout=standby_message_timeout;
488488
stream.synchronous= false;
489-
stream.do_sync=do_sync;
489+
/* fsync happens at the end of pg_basebackup for all data */
490+
stream.do_sync= false;
490491
stream.mark_done= true;
491492
stream.partial_suffix=NULL;
492493
stream.replication_slot=replication_slot;
493494

494495
if (format=='p')
495-
stream.walmethod=CreateWalDirectoryMethod(param->xlog,0,do_sync);
496+
stream.walmethod=CreateWalDirectoryMethod(param->xlog,0,
497+
stream.do_sync);
496498
else
497-
stream.walmethod=CreateWalTarMethod(param->xlog,compresslevel,do_sync);
499+
stream.walmethod=CreateWalTarMethod(param->xlog,compresslevel,
500+
stream.do_sync);
498501

499502
if (!ReceiveXlogStream(param->bgconn,&stream))
500503

@@ -1346,9 +1349,10 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
13461349
if (copybuf!=NULL)
13471350
PQfreemem(copybuf);
13481351

1349-
/* sync the resulting tar file, errors are not considered fatal */
1350-
if (do_sync&&strcmp(basedir,"-")!=0)
1351-
(void)fsync_fname(filename, false);
1352+
/*
1353+
* Do not sync the resulting tar file yet, all files are synced once at
1354+
* the end.
1355+
*/
13521356
}
13531357

13541358

@@ -2138,9 +2142,9 @@ BaseBackup(void)
21382142

21392143
/*
21402144
* Make data persistent on disk once backup is completed. For tar format
2141-
*once syncingthe parent directoryis fine,each tar filecreated per
2142-
*tablespace has been already synced.In plain format, all the data of
2143-
*thebase directory is synced, taking into account all the tablespaces.
2145+
*syncthe parent directoryand all its contents aseach tar filewas not
2146+
*synced after being completed.In plain format, all the data of the
2147+
* base directory is synced, taking into account all the tablespaces.
21442148
* Errors are not considered fatal.
21452149
*/
21462150
if (do_sync)
@@ -2150,7 +2154,7 @@ BaseBackup(void)
21502154
if (format=='t')
21512155
{
21522156
if (strcmp(basedir,"-")!=0)
2153-
(void)fsync_fname(basedir, true);
2157+
(void)fsync_dir_recurse(basedir);
21542158
}
21552159
else
21562160
{

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp