Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit52c32d8

Browse files
committed
Fix handling of files that source server removes during pg_rewind is running.
After processing the filemap to build the list of chunks that will befetched from the source to rewing the target server, it is possible thata file which was previously processed is removed from the source. Asimple example of such an occurence is a WAL segment which gets recycledon the target in-between. When the filemap is processed, files notcategorized as relation files are first truncated to prepare for itsfull copy of which is going to be taken from the source, divided into aset of junks. However, for a recycled WAL segment, this would result ina segment which has a zero-byte size. With such an empty file,post-rewind recovery thinks that records are saved but they are actuallynot because of the truncation which happened when processing thefilemap, resulting in data loss.In order to fix the problem, make sure that files which are found asremoved on the source when receiving chunks of them are as well deletedon the target server for consistency.Back-patch to 9.5 where pg_rewind was added.Author: Tsunakawa TakayukiReviewed-by: Michael PaquierReported-by: Tsunakawa TakayukiDiscussion:https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8DAAA2%40G01JPEXMBYT05
1 parent90decdb commit52c32d8

File tree

3 files changed

+20
-7
lines changed

3 files changed

+20
-7
lines changed

‎src/bin/pg_rewind/file_ops.c

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@
3030
staticintdstfd=-1;
3131
staticchardstpath[MAXPGPATH]="";
3232

33-
staticvoidremove_target_file(constchar*path);
3433
staticvoidcreate_target_dir(constchar*path);
3534
staticvoidremove_target_dir(constchar*path);
3635
staticvoidcreate_target_symlink(constchar*path,constchar*link);
@@ -135,7 +134,7 @@ remove_target(file_entry_t *entry)
135134
break;
136135

137136
caseFILE_TYPE_REGULAR:
138-
remove_target_file(entry->path);
137+
remove_target_file(entry->path, false);
139138
break;
140139

141140
caseFILE_TYPE_SYMLINK:
@@ -166,8 +165,12 @@ create_target(file_entry_t *entry)
166165
}
167166
}
168167

169-
staticvoid
170-
remove_target_file(constchar*path)
168+
/*
169+
* Remove a file from target data directory. If missing_ok is true, it
170+
* is fine for the target file to not exist.
171+
*/
172+
void
173+
remove_target_file(constchar*path,boolmissing_ok)
171174
{
172175
chardstpath[MAXPGPATH];
173176

@@ -176,8 +179,13 @@ remove_target_file(const char *path)
176179

177180
snprintf(dstpath,sizeof(dstpath),"%s/%s",datadir_target,path);
178181
if (unlink(dstpath)!=0)
182+
{
183+
if (errno==ENOENT&&missing_ok)
184+
return;
185+
179186
pg_fatal("could not remove file \"%s\": %s\n",
180187
dstpath,strerror(errno));
188+
}
181189
}
182190

183191
void

‎src/bin/pg_rewind/file_ops.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
externvoidopen_target_file(constchar*path,booltrunc);
1616
externvoidwrite_target_range(char*buf,off_tbegin,size_tsize);
1717
externvoidclose_target_file(void);
18+
externvoidremove_target_file(constchar*path,boolmissing_ok);
1819
externvoidtruncate_target_file(constchar*path,off_tnewsize);
1920
externvoidcreate_target(file_entry_t*t);
2021
externvoidremove_target(file_entry_t*t);

‎src/bin/pg_rewind/libpq_fetch.c

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -337,15 +337,19 @@ receiveFileChunks(const char *sql)
337337
chunk=PQgetvalue(res,0,2);
338338

339339
/*
340-
* It's possible that the file was deleted on remote side after we
341-
* created the file map. In this case simply ignore it, as if it was
342-
* not there in the first place, and move on.
340+
* If a file has been deleted on the source, remove it on the target
341+
* as well. Note that multiple unlink() calls may happen on the same
342+
* file if multiple data chunks are associated with it, hence ignore
343+
* unconditionally anything missing. If this file is not a relation
344+
* data file, then it has been already truncated when creating the
345+
* file chunk list at the previous execution of the filemap.
343346
*/
344347
if (PQgetisnull(res,0,2))
345348
{
346349
pg_log(PG_DEBUG,
347350
"received null value for chunk for file \"%s\", file has been deleted\n",
348351
filename);
352+
remove_target_file(filename, true);
349353
pg_free(filename);
350354
PQclear(res);
351355
continue;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp