- Notifications
You must be signed in to change notification settings - Fork28
Commit6d8727f
committed
Ensure cleanup of orphan archive status files
When a WAL segment is recycled, its ".ready" and ".done" status filesget also automatically removed, however this is not done in a durablemanner. Hence, in a subsequent crash, it could be possible that a".ready" status file is still around with its corresponding segmentalready gone.If the backend reaches such a state, the archive command would mostlikely complain about a segment non-existing and would keep retrying,causing WAL segments to bloat pg_wal/, potentially making Postgres crashhard when running out of space.As status files are removed after each individual segment, usingdurable_unlink() does not completely close the window either, as a crashcould happen between the moment the WAL segment is recycled and themoment its status files are removed. This has also some performanceimpact with the additional fsync() calls needed to make the removal in adurable manner. Doing the cleanup at recovery is not cost-free eitheras this makes crash recovery potentially take longer than necessary.So, instead, as per an idea of Stephen Frost, make the archiver aware oforphan status files and remove them on-the-fly if the correspondingsegment goes missing. Removal failures follow a model close to whathappens for WAL segments, where multiple attempts are done before givingup temporarily, and where a successful orphan removal makes the archivermove immediately to the next WAL segment thought as ready to bearchived.Author: Michael PaquierReviewed-by: Nathan Bossart, Andres Freund, Stephen Frost, KyotaroHoriguchiDiscussion:https://postgr.es/m/20180928032827.GF1500@paquier.xyz1 parent7fee252 commit6d8727f
1 file changed
+55
-0
lines changedLines changed: 55 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
28 | 28 |
| |
29 | 29 |
| |
30 | 30 |
| |
| 31 | + | |
31 | 32 |
| |
32 | 33 |
| |
33 | 34 |
| |
| |||
59 | 60 |
| |
60 | 61 |
| |
61 | 62 |
| |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
62 | 67 |
| |
63 | 68 |
| |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
64 | 75 |
| |
65 | 76 |
| |
66 | 77 |
| |
| |||
424 | 435 |
| |
425 | 436 |
| |
426 | 437 |
| |
| 438 | + | |
427 | 439 |
| |
428 | 440 |
| |
429 | 441 |
| |
| 442 | + | |
| 443 | + | |
| 444 | + | |
430 | 445 |
| |
431 | 446 |
| |
432 | 447 |
| |
| |||
456 | 471 |
| |
457 | 472 |
| |
458 | 473 |
| |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
459 | 514 |
| |
460 | 515 |
| |
461 | 516 |
| |
|
0 commit comments
Comments
(0)