Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitb8fd4e0

Browse files
committed
Adjust max_slot_wal_keep_size behavior per review
In pg_replication_slot, change output from normal/reserved/lost toreserved/extended/unreserved/ lost, which better expresses the possiblestates particularly near the time where segments are no longer safe butcheckpoint has not run yet.Under the new definition, reserved means the slot is consuming WALthat's still under the normal WAL size constraints; extended means it'sconsuming WAL that's being protected by wal_keep_segments or the slotitself, whose size is below max_slot_wal_keep_size; unreserved means theWAL is no longer safe, but checkpoint has not yet removed those files.Such as slot is in imminent danger, but can still continue for a littlewhile and may catch up to the reserved WAL space.Also, there were some bugs in the calculations used to report thestatus; fixed those.Backpatch to 13.Reported-by: Fujii Masao <masao.fujii@oss.nttdata.com>Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>Discussion:https://postgr.es/m/20200616.120236.1809496990963386593.horikyota.ntt@gmail.com
1 parent0188bb8 commitb8fd4e0

File tree

5 files changed

+96
-53
lines changed

5 files changed

+96
-53
lines changed

‎doc/src/sgml/catalogs.sgml

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11239,19 +11239,29 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
1123911239
Possible values are:
1124011240
<itemizedlist>
1124111241
<listitem>
11242-
<para><literal>normal</literal> means that the claimed files
11242+
<para><literal>reserved</literal> means that the claimed files
1124311243
are within <varname>max_wal_size</varname>.</para>
1124411244
</listitem>
1124511245
<listitem>
11246-
<para><literal>reserved</literal> means
11246+
<para><literal>extended</literal> means
1124711247
that <varname>max_wal_size</varname> is exceeded but the files are
11248-
still held, either by some replication slot or
11249-
by <varname>wal_keep_segments</varname>.</para>
11248+
still retained, either by the replication slot or
11249+
by <varname>wal_keep_segments</varname>.
11250+
</para>
1125011251
</listitem>
1125111252
<listitem>
11252-
<para><literal>lost</literal> means that some WAL files are
11253-
definitely lost and this slot cannot be used to resume replication
11254-
anymore.</para>
11253+
<para>
11254+
<literal>unreserved</literal> means that the slot no longer
11255+
retains the required WAL files and some of them are to be removed at
11256+
the next checkpoint. This state can return
11257+
to <literal>reserved</literal> or <literal>extended</literal>.
11258+
</para>
11259+
</listitem>
11260+
<listitem>
11261+
<para>
11262+
<literal>lost</literal> means that some required WAL files have
11263+
been removed and this slot is no longer usable.
11264+
</para>
1125511265
</listitem>
1125611266
</itemizedlist>
1125711267
The last two states are seen only when

‎src/backend/access/transam/xlog.c

Lines changed: 31 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -9488,15 +9488,20 @@ CreateRestartPoint(int flags)
94889488
*(typically a slot's restart_lsn)
94899489
*
94909490
* Returns one of the following enum values:
9491-
* * WALAVAIL_NORMAL means targetLSN is available because it is in the range
9492-
* of max_wal_size.
94939491
*
9494-
* * WALAVAIL_PRESERVED means it is still available by preserving extra
9492+
* * WALAVAIL_RESERVED means targetLSN is available and it is in the range of
9493+
* max_wal_size.
9494+
*
9495+
* * WALAVAIL_EXTENDED means it is still available by preserving extra
94959496
* segments beyond max_wal_size. If max_slot_wal_keep_size is smaller
94969497
* than max_wal_size, this state is not returned.
94979498
*
9498-
* * WALAVAIL_REMOVED means it is definitely lost. A replication stream on
9499-
* a slot with this LSN cannot continue.
9499+
* * WALAVAIL_UNRESERVED means it is being lost and the next checkpoint will
9500+
* remove reserved segments. The walsender using this slot may return to the
9501+
* above.
9502+
*
9503+
* * WALAVAIL_REMOVED means it has been removed. A replication stream on
9504+
* a slot with this LSN cannot continue after a restart.
95009505
*
95019506
* * WALAVAIL_INVALID_LSN means the slot hasn't been set to reserve WAL.
95029507
*/
@@ -9512,13 +9517,18 @@ GetWALAvailability(XLogRecPtr targetLSN)
95129517
* slot */
95139518
uint64keepSegs;
95149519

9515-
/* slot does not reserve WAL. Either deactivated, or has never been active */
9520+
/*
9521+
* slot does not reserve WAL. Either deactivated, or has never been active
9522+
*/
95169523
if (XLogRecPtrIsInvalid(targetLSN))
95179524
returnWALAVAIL_INVALID_LSN;
95189525

95199526
currpos=GetXLogWriteRecPtr();
95209527

9521-
/* calculate oldest segment currently needed by slots */
9528+
/*
9529+
* calculate the oldest segment currently reserved by all slots,
9530+
* considering wal_keep_segments and max_slot_wal_keep_size
9531+
*/
95229532
XLByteToSeg(targetLSN,targetSeg,wal_segment_size);
95239533
KeepLogSeg(currpos,&oldestSlotSeg);
95249534

@@ -9529,38 +9539,33 @@ GetWALAvailability(XLogRecPtr targetLSN)
95299539
*/
95309540
oldestSeg=XLogGetLastRemovedSegno()+1;
95319541

9532-
/* calculate oldest segment by max_wal_sizeand wal_keep_segments*/
9542+
/* calculate oldest segment by max_wal_size */
95339543
XLByteToSeg(currpos,currSeg,wal_segment_size);
9534-
keepSegs=ConvertToXSegs(Max(max_wal_size_mb,wal_keep_segments),
9535-
wal_segment_size)+1;
9544+
keepSegs=ConvertToXSegs(max_wal_size_mb,wal_segment_size)+1;
95369545

95379546
if (currSeg>keepSegs)
95389547
oldestSegMaxWalSize=currSeg-keepSegs;
95399548
else
95409549
oldestSegMaxWalSize=1;
95419550

95429551
/*
9543-
* If max_slot_wal_keep_size has changed after the last call, the segment
9544-
* that would been kept by the current setting might have been lost by the
9545-
* previous setting. No point in showing normal or keeping status values
9546-
* if the targetSeg is known to be lost.
9552+
* No point in returning reserved or extended status values if the
9553+
* targetSeg is known to be lost.
95479554
*/
9548-
if (targetSeg >=oldestSeg)
9555+
if (targetSeg >=oldestSlotSeg)
95499556
{
9550-
/*
9551-
* show "normal" when targetSeg is within max_wal_size, even if
9552-
* max_slot_wal_keep_size is smaller than max_wal_size.
9553-
*/
9554-
if ((max_slot_wal_keep_size_mb <=0||
9555-
max_slot_wal_keep_size_mb >=max_wal_size_mb)&&
9556-
oldestSegMaxWalSize <=targetSeg)
9557-
returnWALAVAIL_NORMAL;
9558-
9559-
/* being retained by slots */
9560-
if (oldestSlotSeg <=targetSeg)
9557+
/* show "reserved" when targetSeg is within max_wal_size */
9558+
if (targetSeg >=oldestSegMaxWalSize)
95619559
returnWALAVAIL_RESERVED;
9560+
9561+
/* being retained by slots exceeding max_wal_size */
9562+
returnWALAVAIL_EXTENDED;
95629563
}
95639564

9565+
/* WAL segments are no longer retained but haven't been removed yet */
9566+
if (targetSeg >=oldestSeg)
9567+
returnWALAVAIL_UNRESERVED;
9568+
95649569
/* Definitely lost */
95659570
returnWALAVAIL_REMOVED;
95669571
}

‎src/backend/replication/slotfuncs.c

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -359,24 +359,47 @@ pg_get_replication_slots(PG_FUNCTION_ARGS)
359359
nulls[i++]= true;
360360
break;
361361

362-
caseWALAVAIL_NORMAL:
363-
values[i++]=CStringGetTextDatum("normal");
364-
break;
365-
366362
caseWALAVAIL_RESERVED:
367363
values[i++]=CStringGetTextDatum("reserved");
368364
break;
369365

366+
caseWALAVAIL_EXTENDED:
367+
values[i++]=CStringGetTextDatum("extended");
368+
break;
369+
370+
caseWALAVAIL_UNRESERVED:
371+
values[i++]=CStringGetTextDatum("unreserved");
372+
break;
373+
370374
caseWALAVAIL_REMOVED:
375+
376+
/*
377+
* If we read the restart_lsn long enough ago, maybe that file
378+
* has been removed by now. However, the walsender could have
379+
* moved forward enough that it jumped to another file after
380+
* we looked. If checkpointer signalled the process to
381+
* termination, then it's definitely lost; but if a process is
382+
* still alive, then "unreserved" seems more appropriate.
383+
*/
384+
if (!XLogRecPtrIsInvalid(slot_contents.data.restart_lsn))
385+
{
386+
intpid;
387+
388+
SpinLockAcquire(&slot->mutex);
389+
pid=slot->active_pid;
390+
SpinLockRelease(&slot->mutex);
391+
if (pid!=0)
392+
{
393+
values[i++]=CStringGetTextDatum("unreserved");
394+
break;
395+
}
396+
}
371397
values[i++]=CStringGetTextDatum("lost");
372398
break;
373-
374-
default:
375-
elog(ERROR,"invalid walstate: %d", (int)walstate);
376399
}
377400

378401
if (max_slot_wal_keep_size_mb >=0&&
379-
(walstate==WALAVAIL_NORMAL||walstate==WALAVAIL_RESERVED)&&
402+
(walstate==WALAVAIL_RESERVED||walstate==WALAVAIL_EXTENDED)&&
380403
((last_removed_seg=XLogGetLastRemovedSegno())!=0))
381404
{
382405
XLogRecPtrmin_safe_lsn;

‎src/include/access/xlog.h

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -270,8 +270,10 @@ extern CheckpointStatsData CheckpointStats;
270270
typedefenumWALAvailability
271271
{
272272
WALAVAIL_INVALID_LSN,/* parameter error */
273-
WALAVAIL_NORMAL,/* WAL segment is within max_wal_size */
274-
WALAVAIL_RESERVED,/* WAL segment is reserved by a slot */
273+
WALAVAIL_RESERVED,/* WAL segment is within max_wal_size */
274+
WALAVAIL_EXTENDED,/* WAL segment is reserved by a slot or
275+
* wal_keep_segments */
276+
WALAVAIL_UNRESERVED,/* no longer reserved, but not removed yet */
275277
WALAVAIL_REMOVED/* WAL segment has been removed */
276278
}WALAvailability;
277279

‎src/test/recovery/t/019_replslot_limit.pl

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@
5656
$result =$node_master->safe_psql('postgres',
5757
"SELECT wal_status, min_safe_lsn is NULL FROM pg_replication_slots WHERE slot_name = 'rep1'"
5858
);
59-
is($result,"normal|t",'check the catching-up state');
59+
is($result,"reserved|t",'check the catching-up state');
6060

6161
# Advance WAL by five segments (= 5MB) on master
6262
advance_wal($node_master, 1);
@@ -66,7 +66,8 @@
6666
$result =$node_master->safe_psql('postgres',
6767
"SELECT wal_status, min_safe_lsn is NULL FROM pg_replication_slots WHERE slot_name = 'rep1'"
6868
);
69-
is($result,"normal|t",'check that it is safe if WAL fits in max_wal_size');
69+
is($result,"reserved|t",
70+
'check that it is safe if WAL fits in max_wal_size');
7071

7172
advance_wal($node_master, 4);
7273
$node_master->safe_psql('postgres',"CHECKPOINT;");
@@ -75,7 +76,7 @@
7576
$result =$node_master->safe_psql('postgres',
7677
"SELECT wal_status, min_safe_lsn is NULL FROM pg_replication_slots WHERE slot_name = 'rep1'"
7778
);
78-
is($result,"normal|t",'check that slot is working');
79+
is($result,"reserved|t",'check that slot is working');
7980

8081
# The standby can reconnect to master
8182
$node_standby->start;
@@ -99,7 +100,7 @@
99100

100101
$result =$node_master->safe_psql('postgres',
101102
"SELECT wal_status FROM pg_replication_slots WHERE slot_name = 'rep1'");
102-
is($result,"normal",'check that max_slot_wal_keep_size is working');
103+
is($result,"reserved",'check that max_slot_wal_keep_size is working');
103104

104105
# Advance WAL again then checkpoint, reducing remain by 2 MB.
105106
advance_wal($node_master, 2);
@@ -108,7 +109,7 @@
108109
# The slot is still working
109110
$result =$node_master->safe_psql('postgres',
110111
"SELECT wal_status FROM pg_replication_slots WHERE slot_name = 'rep1'");
111-
is($result,"normal",
112+
is($result,"reserved",
112113
'check that min_safe_lsn gets close to the current LSN');
113114

114115
# The standby can reconnect to master
@@ -125,7 +126,7 @@
125126
$result =$node_master->safe_psql('postgres',
126127
"SELECT wal_status as remain FROM pg_replication_slots WHERE slot_name = 'rep1'"
127128
);
128-
is($result,"normal",
129+
is($result,"extended",
129130
'check that wal_keep_segments overrides max_slot_wal_keep_size');
130131
# restore wal_keep_segments
131132
$result =$node_master->safe_psql('postgres',
@@ -143,19 +144,20 @@
143144
# Slot gets into 'reserved' state
144145
$result =$node_master->safe_psql('postgres',
145146
"SELECT wal_status FROM pg_replication_slots WHERE slot_name = 'rep1'");
146-
is($result,"reserved",'check that the slot state changes to "reserved"');
147+
is($result,"extended",'check that the slot state changes to "extended"');
147148

148149
# do checkpoint so that the next checkpoint runs too early
149150
$node_master->safe_psql('postgres',"CHECKPOINT;");
150151

151152
# Advance WAL again without checkpoint; remain goes to 0.
152153
advance_wal($node_master, 1);
153154

154-
# Slot gets into 'lost' state
155+
# Slot gets into 'unreserved' state
155156
$result =$node_master->safe_psql('postgres',
156157
"SELECT wal_status, min_safe_lsn is NULL FROM pg_replication_slots WHERE slot_name = 'rep1'"
157158
);
158-
is($result,"lost|t",'check that the slot state changes to "lost"');
159+
is($result,"unreserved|t",
160+
'check that the slot state changes to "unreserved"');
159161

160162
# The standby still can connect to master before a checkpoint
161163
$node_standby->start;
@@ -186,7 +188,8 @@
186188
$result =$node_master->safe_psql('postgres',
187189
"SELECT slot_name, active, restart_lsn IS NULL, wal_status, min_safe_lsn FROM pg_replication_slots WHERE slot_name = 'rep1'"
188190
);
189-
is($result,"rep1|f|t|lost|",'check that the slot became inactive');
191+
is($result,"rep1|f|t|lost|",
192+
'check that the slot became inactive and the state "lost" persists');
190193

191194
# The standby no longer can connect to the master
192195
$logstart = get_log_size($node_standby);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp