Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit70b4f82

Browse files
committed
Prevent hard failures of standbys caused by recycled WAL segments
When a standby's WAL receiver stops reading WAL from a WAL stream, itwrites data to the current WAL segment without having priorily zero'edthe page currently written to, which can cause the WAL reader to readjunk data from a past recycled segment and then it would try to get arecord from it. While sanity checks in place provide most of theprotection needed, in some rare circumstances, with chances increasingwhen a record header crosses a page boundary, then the startup processcould fail violently on an allocation failure, as follows:FATAL: invalid memory alloc request size XXXThis is confusing for the user and also unhelpful as this requires inthe worst case a manual restart of the instance, impacting potentiallythe availability of the cluster, and this also makes WAL data look likeit is in a corrupted state.The chances of seeing failures are higher if the connection between thestandby and its root node is unstable, causing WAL pages to be writtenin the middle. A couple of approaches have been discussed, likezero-ing new WAL pages within the WAL receiver itself but this has thedisadvantage of impacting performance of any existing instances as thisbreaks the sequential writes done by the WAL receiver. This commitdeals with the problem with a more simple approach, which has noperformance impact without reducing the detection of the problem: if arecord is found with a length higher than 1GB for backends, then do nottry any allocation and report a soft failure which will force thestandby to retry reading WAL. It could be possible that the allocationcall passes and that an unnecessary amount of memory is allocated,however follow-up checks on records would just fail, making thisallocation short-lived anyway.This patch owes a great deal to Tsunakawa Takayuki for reporting thefailure first, and then discussing a couple of potential approaches tothe problem.Backpatch down to 9.5, which is where palloc_extended has beenintroduced.Reported-by: Tsunakawa TakayukiReviewed-by: Tsunakawa TakayukiAuthor: Michael PaquierDiscussion:https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8B57AD@G01JPEXMBYT05
1 parent9b53d96 commit70b4f82

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

‎src/backend/access/transam/xlogreader.c

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@
2525
#include"common/pg_lzcompress.h"
2626
#include"replication/origin.h"
2727

28+
#ifndefFRONTEND
29+
#include"utils/memutils.h"
30+
#endif
31+
2832
staticboolallocate_recordbuf(XLogReaderState*state,uint32reclength);
2933

3034
staticboolValidXLogRecordHeader(XLogReaderState*state,XLogRecPtrRecPtr,
@@ -160,6 +164,25 @@ allocate_recordbuf(XLogReaderState *state, uint32 reclength)
160164
newSize+=XLOG_BLCKSZ- (newSize %XLOG_BLCKSZ);
161165
newSize=Max(newSize,5*Max(BLCKSZ,XLOG_BLCKSZ));
162166

167+
#ifndefFRONTEND
168+
169+
/*
170+
* Note that in much unlucky circumstances, the random data read from a
171+
* recycled segment can cause this routine to be called with a size
172+
* causing a hard failure at allocation. For a standby, this would cause
173+
* the instance to stop suddenly with a hard failure, preventing it to
174+
* retry fetching WAL from one of its sources which could allow it to move
175+
* on with replay without a manual restart. If the data comes from a past
176+
* recycled segment and is still valid, then the allocation may succeed
177+
* but record checks are going to fail so this would be short-lived. If
178+
* the allocation fails because of a memory shortage, then this is not a
179+
* hard failure either per the guarantee given by MCXT_ALLOC_NO_OOM.
180+
*/
181+
if (!AllocSizeIsValid(newSize))
182+
return false;
183+
184+
#endif
185+
163186
if (state->readRecordBuf)
164187
pfree(state->readRecordBuf);
165188
state->readRecordBuf=

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp