Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit09e96b3

Browse files
committed
Fix handling of files that source server removes during pg_rewind is running.
After processing the filemap to build the list of chunks that will befetched from the source to rewing the target server, it is possible thata file which was previously processed is removed from the source. Asimple example of such an occurence is a WAL segment which gets recycledon the target in-between. When the filemap is processed, files notcategorized as relation files are first truncated to prepare for itsfull copy of which is going to be taken from the source, divided into aset of junks. However, for a recycled WAL segment, this would result ina segment which has a zero-byte size. With such an empty file,post-rewind recovery thinks that records are saved but they are actuallynot because of the truncation which happened when processing thefilemap, resulting in data loss.In order to fix the problem, make sure that files which are found asremoved on the source when receiving chunks of them are as well deletedon the target server for consistency.Back-patch to 9.5 where pg_rewind was added.Author: Tsunakawa TakayukiReviewed-by: Michael PaquierReported-by: Tsunakawa TakayukiDiscussion:https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8DAAA2%40G01JPEXMBYT05
1 parentd92bc83 commit09e96b3

File tree

3 files changed

+20
-7
lines changed

3 files changed

+20
-7
lines changed

‎src/bin/pg_rewind/file_ops.c

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@
2929
staticintdstfd=-1;
3030
staticchardstpath[MAXPGPATH]="";
3131

32-
staticvoidremove_target_file(constchar*path);
3332
staticvoidcreate_target_dir(constchar*path);
3433
staticvoidremove_target_dir(constchar*path);
3534
staticvoidcreate_target_symlink(constchar*path,constchar*link);
@@ -134,7 +133,7 @@ remove_target(file_entry_t *entry)
134133
break;
135134

136135
caseFILE_TYPE_REGULAR:
137-
remove_target_file(entry->path);
136+
remove_target_file(entry->path, false);
138137
break;
139138

140139
caseFILE_TYPE_SYMLINK:
@@ -165,8 +164,12 @@ create_target(file_entry_t *entry)
165164
}
166165
}
167166

168-
staticvoid
169-
remove_target_file(constchar*path)
167+
/*
168+
* Remove a file from target data directory. If missing_ok is true, it
169+
* is fine for the target file to not exist.
170+
*/
171+
void
172+
remove_target_file(constchar*path,boolmissing_ok)
170173
{
171174
chardstpath[MAXPGPATH];
172175

@@ -175,8 +178,13 @@ remove_target_file(const char *path)
175178

176179
snprintf(dstpath,sizeof(dstpath),"%s/%s",datadir_target,path);
177180
if (unlink(dstpath)!=0)
181+
{
182+
if (errno==ENOENT&&missing_ok)
183+
return;
184+
178185
pg_fatal("could not remove file \"%s\": %s\n",
179186
dstpath,strerror(errno));
187+
}
180188
}
181189

182190
void

‎src/bin/pg_rewind/file_ops.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
externvoidopen_target_file(constchar*path,booltrunc);
1616
externvoidwrite_target_range(char*buf,off_tbegin,size_tsize);
1717
externvoidclose_target_file(void);
18+
externvoidremove_target_file(constchar*path,boolmissing_ok);
1819
externvoidtruncate_target_file(constchar*path,off_tnewsize);
1920
externvoidcreate_target(file_entry_t*t);
2021
externvoidremove_target(file_entry_t*t);

‎src/bin/pg_rewind/libpq_fetch.c

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -311,15 +311,19 @@ receiveFileChunks(const char *sql)
311311
chunk=PQgetvalue(res,0,2);
312312

313313
/*
314-
* It's possible that the file was deleted on remote side after we
315-
* created the file map. In this case simply ignore it, as if it was
316-
* not there in the first place, and move on.
314+
* If a file has been deleted on the source, remove it on the target
315+
* as well. Note that multiple unlink() calls may happen on the same
316+
* file if multiple data chunks are associated with it, hence ignore
317+
* unconditionally anything missing. If this file is not a relation
318+
* data file, then it has been already truncated when creating the
319+
* file chunk list at the previous execution of the filemap.
317320
*/
318321
if (PQgetisnull(res,0,2))
319322
{
320323
pg_log(PG_DEBUG,
321324
"received null value for chunk for file \"%s\", file has been deleted\n",
322325
filename);
326+
remove_target_file(filename, true);
323327
pg_free(filename);
324328
PQclear(res);
325329
continue;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp