Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitf1e07d5

Browse files
committed
Fix handling of files that source server removes during pg_rewind is running.
After processing the filemap to build the list of chunks that will befetched from the source to rewing the target server, it is possible thata file which was previously processed is removed from the source. Asimple example of such an occurence is a WAL segment which gets recycledon the target in-between. When the filemap is processed, files notcategorized as relation files are first truncated to prepare for itsfull copy of which is going to be taken from the source, divided into aset of junks. However, for a recycled WAL segment, this would result ina segment which has a zero-byte size. With such an empty file,post-rewind recovery thinks that records are saved but they are actuallynot because of the truncation which happened when processing thefilemap, resulting in data loss.In order to fix the problem, make sure that files which are found asremoved on the source when receiving chunks of them are as well deletedon the target server for consistency.Back-patch to 9.5 where pg_rewind was added.Author: Tsunakawa TakayukiReviewed-by: Michael PaquierReported-by: Tsunakawa TakayukiDiscussion:https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8DAAA2%40G01JPEXMBYT05
1 parentc98f218 commitf1e07d5

File tree

3 files changed

+20
-7
lines changed

3 files changed

+20
-7
lines changed

‎src/bin/pg_rewind/file_ops.c

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@
2929
staticintdstfd=-1;
3030
staticchardstpath[MAXPGPATH]="";
3131

32-
staticvoidremove_target_file(constchar*path);
3332
staticvoidcreate_target_dir(constchar*path);
3433
staticvoidremove_target_dir(constchar*path);
3534
staticvoidcreate_target_symlink(constchar*path,constchar*link);
@@ -134,7 +133,7 @@ remove_target(file_entry_t *entry)
134133
break;
135134

136135
caseFILE_TYPE_REGULAR:
137-
remove_target_file(entry->path);
136+
remove_target_file(entry->path, false);
138137
break;
139138

140139
caseFILE_TYPE_SYMLINK:
@@ -165,8 +164,12 @@ create_target(file_entry_t *entry)
165164
}
166165
}
167166

168-
staticvoid
169-
remove_target_file(constchar*path)
167+
/*
168+
* Remove a file from target data directory. If missing_ok is true, it
169+
* is fine for the target file to not exist.
170+
*/
171+
void
172+
remove_target_file(constchar*path,boolmissing_ok)
170173
{
171174
chardstpath[MAXPGPATH];
172175

@@ -175,8 +178,13 @@ remove_target_file(const char *path)
175178

176179
snprintf(dstpath,sizeof(dstpath),"%s/%s",datadir_target,path);
177180
if (unlink(dstpath)!=0)
181+
{
182+
if (errno==ENOENT&&missing_ok)
183+
return;
184+
178185
pg_fatal("could not remove file \"%s\": %s\n",
179186
dstpath,strerror(errno));
187+
}
180188
}
181189

182190
void

‎src/bin/pg_rewind/file_ops.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
externvoidopen_target_file(constchar*path,booltrunc);
1616
externvoidwrite_target_range(char*buf,off_tbegin,size_tsize);
1717
externvoidclose_target_file(void);
18+
externvoidremove_target_file(constchar*path,boolmissing_ok);
1819
externvoidtruncate_target_file(constchar*path,off_tnewsize);
1920
externvoidcreate_target(file_entry_t*t);
2021
externvoidremove_target(file_entry_t*t);

‎src/bin/pg_rewind/libpq_fetch.c

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -336,15 +336,19 @@ receiveFileChunks(const char *sql)
336336
chunk=PQgetvalue(res,0,2);
337337

338338
/*
339-
* It's possible that the file was deleted on remote side after we
340-
* created the file map. In this case simply ignore it, as if it was
341-
* not there in the first place, and move on.
339+
* If a file has been deleted on the source, remove it on the target
340+
* as well. Note that multiple unlink() calls may happen on the same
341+
* file if multiple data chunks are associated with it, hence ignore
342+
* unconditionally anything missing. If this file is not a relation
343+
* data file, then it has been already truncated when creating the
344+
* file chunk list at the previous execution of the filemap.
342345
*/
343346
if (PQgetisnull(res,0,2))
344347
{
345348
pg_log(PG_DEBUG,
346349
"received null value for chunk for file \"%s\", file has been deleted\n",
347350
filename);
351+
remove_target_file(filename, true);
348352
pg_free(filename);
349353
PQclear(res);
350354
continue;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp