Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit2fbdebc

Browse files
committed
Revert "Skip WAL for new relfilenodes, under wal_level=minimal."
This reverts commitcb2fd7e. Pernumerous buildfarm members, it was incompatible with parallel query, anda test case assumed LP64. Back-patch to 9.5 (all supported versions).Discussion:https://postgr.es/m/20200321224920.GB1763544@rfd.leadboat.com
1 parent03b89f1 commit2fbdebc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+337
-1404
lines changed

‎doc/src/sgml/config.sgml

Lines changed: 8 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -2323,19 +2323,16 @@ include_dir 'conf.d'
23232323
levels. This parameter can only be set at server start.
23242324
</para>
23252325
<para>
2326-
In <literal>minimal</literal> level, no information is logged for
2327-
permanent relations for the remainder of a transaction that creates or
2328-
rewrites them. This can make operations much faster (see
2329-
<xref linkend="populate-pitr"/>). Operations that initiate this
2330-
optimization include:
2326+
In <literal>minimal</literal> level, WAL-logging of some bulk
2327+
operations can be safely skipped, which can make those
2328+
operations much faster (see <xref linkend="populate-pitr"/>).
2329+
Operations in which this optimization can be applied include:
23312330
<simplelist>
2332-
<member><command>ALTER ... SET TABLESPACE</command></member>
2331+
<member><command>CREATE TABLE AS</command></member>
2332+
<member><command>CREATE INDEX</command></member>
23332333
<member><command>CLUSTER</command></member>
2334-
<member><command>CREATE TABLE</command></member>
2335-
<member><command>REFRESH MATERIALIZED VIEW</command>
2336-
(without <option>CONCURRENTLY</option>)</member>
2337-
<member><command>REINDEX</command></member>
2338-
<member><command>TRUNCATE</command></member>
2334+
<member><command>COPY</command> into tables that were created or truncated in the same
2335+
transaction</member>
23392336
</simplelist>
23402337
But minimal WAL does not contain enough information to reconstruct the
23412338
data from a base backup and the WAL logs, so <literal>replica</literal> or
@@ -2724,26 +2721,6 @@ include_dir 'conf.d'
27242721
</listitem>
27252722
</varlistentry>
27262723

2727-
<varlistentry id="guc-wal-skip-threshold" xreflabel="wal_skip_threshold">
2728-
<term><varname>wal_skip_threshold</varname> (<type>integer</type>)
2729-
<indexterm>
2730-
<primary><varname>wal_skip_threshold</varname> configuration parameter</primary>
2731-
</indexterm>
2732-
</term>
2733-
<listitem>
2734-
<para>
2735-
When <varname>wal_level</varname> is <literal>minimal</literal> and a
2736-
transaction commits after creating or rewriting a permanent relation,
2737-
this setting determines how to persist the new data. If the data is
2738-
smaller than this setting, write it to the WAL log; otherwise, use an
2739-
fsync of affected files. Depending on the properties of your storage,
2740-
raising or lowering this value might help if such commits are slowing
2741-
concurrent transactions. The default is two megabytes
2742-
(<literal>2MB</literal>).
2743-
</para>
2744-
</listitem>
2745-
</varlistentry>
2746-
27472724
<varlistentry id="guc-commit-delay" xreflabel="commit_delay">
27482725
<term><varname>commit_delay</varname> (<type>integer</type>)
27492726
<indexterm>

‎doc/src/sgml/perform.sgml

Lines changed: 38 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1536,8 +1536,8 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse;
15361536
needs to be written, because in case of an error, the files
15371537
containing the newly loaded data will be removed anyway.
15381538
However, this consideration only applies when
1539-
<xref linkend="guc-wal-level"/> is <literal>minimal</literal>
1540-
as all commands must write WAL otherwise.
1539+
<xref linkend="guc-wal-level"/> is <literal>minimal</literal> for
1540+
non-partitioned tablesas all commands must write WAL otherwise.
15411541
</para>
15421542

15431543
</sect2>
@@ -1637,13 +1637,42 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse;
16371637
</para>
16381638

16391639
<para>
1640-
Aside from avoiding the time for the archiver or WAL sender to process the
1641-
WAL data, doing this will actually make certain commands faster, because
1642-
they do not to write WAL at all if <varname>wal_level</varname>
1643-
is <literal>minimal</literal> and the current subtransaction (or top-level
1644-
transaction) created or truncated the table or index they change. (They
1645-
can guarantee crash safety more cheaply by doing
1646-
an <function>fsync</function> at the end than by writing WAL.)
1640+
Aside from avoiding the time for the archiver or WAL sender to
1641+
process the WAL data,
1642+
doing this will actually make certain commands faster, because they
1643+
are designed not to write WAL at all if <varname>wal_level</varname>
1644+
is <literal>minimal</literal>. (They can guarantee crash safety more cheaply
1645+
by doing an <function>fsync</function> at the end than by writing WAL.)
1646+
This applies to the following commands:
1647+
<itemizedlist>
1648+
<listitem>
1649+
<para>
1650+
<command>CREATE TABLE AS SELECT</command>
1651+
</para>
1652+
</listitem>
1653+
<listitem>
1654+
<para>
1655+
<command>CREATE INDEX</command> (and variants such as
1656+
<command>ALTER TABLE ADD PRIMARY KEY</command>)
1657+
</para>
1658+
</listitem>
1659+
<listitem>
1660+
<para>
1661+
<command>ALTER TABLE SET TABLESPACE</command>
1662+
</para>
1663+
</listitem>
1664+
<listitem>
1665+
<para>
1666+
<command>CLUSTER</command>
1667+
</para>
1668+
</listitem>
1669+
<listitem>
1670+
<para>
1671+
<command>COPY FROM</command>, when the target table has been
1672+
created or truncated earlier in the same transaction
1673+
</para>
1674+
</listitem>
1675+
</itemizedlist>
16471676
</para>
16481677
</sect2>
16491678

‎src/backend/access/gist/gistbuild.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -191,7 +191,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo)
191191
PageSetLSN(page,recptr);
192192
}
193193
else
194-
PageSetLSN(page,gistGetFakeLSN(index));
194+
PageSetLSN(page,gistGetFakeLSN(heap));
195195

196196
UnlockReleaseBuffer(buffer);
197197

‎src/backend/access/gist/gistutil.c

Lines changed: 5 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -972,44 +972,23 @@ gistproperty(Oid index_oid, int attno,
972972
}
973973

974974
/*
975-
*Someindexes are not WAL-logged, but we need LSNs to detect concurrent page
976-
* splits anyway. This function provides a fake sequence of LSNs for that
977-
* purpose.
975+
*Temporary and unlogged GiSTindexes are not WAL-logged, but we need LSNs
976+
*to detect concurrent pagesplits anyway. This function provides a fake
977+
*sequence of LSNs for thatpurpose.
978978
*/
979979
XLogRecPtr
980980
gistGetFakeLSN(Relationrel)
981981
{
982+
staticXLogRecPtrcounter=1;
983+
982984
if (rel->rd_rel->relpersistence==RELPERSISTENCE_TEMP)
983985
{
984986
/*
985987
* Temporary relations are only accessible in our session, so a simple
986988
* backend-local counter will do.
987989
*/
988-
staticXLogRecPtrcounter=1;
989-
990990
returncounter++;
991991
}
992-
elseif (rel->rd_rel->relpersistence==RELPERSISTENCE_PERMANENT)
993-
{
994-
/*
995-
* WAL-logging on this relation will start after commit, so its LSNs
996-
* must be distinct numbers smaller than the LSN at the next commit.
997-
* Emit a dummy WAL record if insert-LSN hasn't advanced after the
998-
* last call.
999-
*/
1000-
staticXLogRecPtrlastlsn=InvalidXLogRecPtr;
1001-
XLogRecPtrcurrlsn=GetXLogInsertRecPtr();
1002-
1003-
/* Shouldn't be called for WAL-logging relations */
1004-
Assert(!RelationNeedsWAL(rel));
1005-
1006-
/* No need for an actual record if we already have a distinct LSN */
1007-
if (!XLogRecPtrIsInvalid(lastlsn)&&lastlsn==currlsn)
1008-
currlsn=gistXLogAssignLSN();
1009-
1010-
lastlsn=currlsn;
1011-
returncurrlsn;
1012-
}
1013992
else
1014993
{
1015994
/*

‎src/backend/access/gist/gistxlog.c

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -505,9 +505,6 @@ gist_redo(XLogReaderState *record)
505505
caseXLOG_GIST_CREATE_INDEX:
506506
gistRedoCreateIndex(record);
507507
break;
508-
caseXLOG_GIST_ASSIGN_LSN:
509-
/* nop. See gistGetFakeLSN(). */
510-
break;
511508
default:
512509
elog(PANIC,"gist_redo: unknown op code %u",info);
513510
}
@@ -626,24 +623,6 @@ gistXLogSplit(bool page_is_leaf,
626623
returnrecptr;
627624
}
628625

629-
/*
630-
* Write an empty XLOG record to assign a distinct LSN.
631-
*/
632-
XLogRecPtr
633-
gistXLogAssignLSN(void)
634-
{
635-
intdummy=0;
636-
637-
/*
638-
* Records other than SWITCH_WAL must have content. We use an integer 0 to
639-
* follow the restriction.
640-
*/
641-
XLogBeginInsert();
642-
XLogSetRecordFlags(XLOG_MARK_UNIMPORTANT);
643-
XLogRegisterData((char*)&dummy,sizeof(dummy));
644-
returnXLogInsert(RM_GIST_ID,XLOG_GIST_ASSIGN_LSN);
645-
}
646-
647626
/*
648627
* Write XLOG record describing a page update. The update can include any
649628
* number of deletions and/or insertions of tuples on a single index page.

‎src/backend/access/heap/heapam.c

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
*heap_multi_insert - insert multiple tuples into a relation
2828
*heap_delete- delete a tuple from a relation
2929
*heap_update- replace a tuple in a relation with another tuple
30+
*heap_sync- sync heap, for when no WAL has been written
3031
*
3132
* NOTES
3233
* This file contains the heap_ routines which implement
@@ -2395,6 +2396,12 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
23952396
* The new tuple is stamped with current transaction ID and the specified
23962397
* command ID.
23972398
*
2399+
* If the HEAP_INSERT_SKIP_WAL option is specified, the new tuple is not
2400+
* logged in WAL, even for a non-temp relation. Safe usage of this behavior
2401+
* requires that we arrange that all new tuples go into new pages not
2402+
* containing any tuples from other transactions, and that the relation gets
2403+
* fsync'd before commit. (See also heap_sync() comments)
2404+
*
23982405
* The HEAP_INSERT_SKIP_FSM option is passed directly to
23992406
* RelationGetBufferForTuple, which see for more info.
24002407
*
@@ -2503,7 +2510,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
25032510
MarkBufferDirty(buffer);
25042511

25052512
/* XLOG stuff */
2506-
if (RelationNeedsWAL(relation))
2513+
if (!(options&HEAP_INSERT_SKIP_WAL)&&RelationNeedsWAL(relation))
25072514
{
25082515
xl_heap_insertxlrec;
25092516
xl_heap_headerxlhdr;
@@ -2713,7 +2720,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
27132720
/* currently not needed (thus unsupported) for heap_multi_insert() */
27142721
AssertArg(!(options&HEAP_INSERT_NO_LOGICAL));
27152722

2716-
needwal=RelationNeedsWAL(relation);
2723+
needwal=!(options&HEAP_INSERT_SKIP_WAL)&&RelationNeedsWAL(relation);
27172724
saveFreeSpace=RelationGetTargetPageFreeSpace(relation,
27182725
HEAP_DEFAULT_FILLFACTOR);
27192726

@@ -9413,13 +9420,18 @@ heap2_redo(XLogReaderState *record)
94139420
}
94149421

94159422
/*
9416-
*heap_sync- for binary compatibility
9417-
*
9418-
* A newer PostgreSQL version removes this function. It exists here just in
9419-
* case an extension calls it. See "Skipping WAL for New RelFileNode" in
9420-
* src/backend/access/transam/README for the system that superseded it,
9421-
* allowing removal of most calls. Cases like copy_relation_data() should
9422-
* call smgrimmedsync() directly.
9423+
*heap_sync- sync a heap, for use when no WAL has been written
9424+
*
9425+
* This forces the heap contents (including TOAST heap if any) down to disk.
9426+
* If we skipped using WAL, and WAL is otherwise needed, we must force the
9427+
* relation down to disk before it's safe to commit the transaction. This
9428+
* requires writing out any dirty buffers and then doing a forced fsync.
9429+
*
9430+
* Indexes are not touched. (Currently, index operations associated with
9431+
* the commands that use this are WAL-logged and so do not need fsync.
9432+
* That behavior might change someday, but in any case it's likely that
9433+
* any fsync decisions required would be per-index and hence not appropriate
9434+
* to be done here.)
94239435
*/
94249436
void
94259437
heap_sync(Relationrel)

‎src/backend/access/heap/rewriteheap.c

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,7 @@ typedef struct RewriteStateData
145145
Pagers_buffer;/* page currently being built */
146146
BlockNumberrs_blockno;/* block where page will go */
147147
boolrs_buffer_valid;/* T if any tuples in buffer */
148+
boolrs_use_wal;/* must we WAL-log inserts? */
148149
boolrs_logical_rewrite;/* do we need to do logical rewriting */
149150
TransactionIdrs_oldest_xmin;/* oldest xmin used by caller to determine
150151
* tuple visibility */
@@ -238,13 +239,15 @@ static void logical_end_heap_rewrite(RewriteState state);
238239
* oldest_xminxid used by the caller to determine which tuples are dead
239240
* freeze_xidxid before which tuples will be frozen
240241
* min_multimultixact before which multis will be removed
242+
* use_walshould the inserts to the new heap be WAL-logged?
241243
*
242244
* Returns an opaque RewriteState, allocated in current memory context,
243245
* to be used in subsequent calls to the other functions.
244246
*/
245247
RewriteState
246248
begin_heap_rewrite(Relationold_heap,Relationnew_heap,TransactionIdoldest_xmin,
247-
TransactionIdfreeze_xid,MultiXactIdcutoff_multi)
249+
TransactionIdfreeze_xid,MultiXactIdcutoff_multi,
250+
booluse_wal)
248251
{
249252
RewriteStatestate;
250253
MemoryContextrw_cxt;
@@ -269,6 +272,7 @@ begin_heap_rewrite(Relation old_heap, Relation new_heap, TransactionId oldest_xm
269272
/* new_heap needn't be empty, just locked */
270273
state->rs_blockno=RelationGetNumberOfBlocks(new_heap);
271274
state->rs_buffer_valid= false;
275+
state->rs_use_wal=use_wal;
272276
state->rs_oldest_xmin=oldest_xmin;
273277
state->rs_freeze_xid=freeze_xid;
274278
state->rs_cutoff_multi=cutoff_multi;
@@ -327,7 +331,7 @@ end_heap_rewrite(RewriteState state)
327331
/* Write the last page, if any */
328332
if (state->rs_buffer_valid)
329333
{
330-
if (RelationNeedsWAL(state->rs_new_rel))
334+
if (state->rs_use_wal)
331335
log_newpage(&state->rs_new_rel->rd_node,
332336
MAIN_FORKNUM,
333337
state->rs_blockno,
@@ -342,14 +346,18 @@ end_heap_rewrite(RewriteState state)
342346
}
343347

344348
/*
345-
* When we WAL-logged rel pages, we must nonetheless fsync them. The
349+
* If the rel is WAL-logged, must fsync before commit. We use heap_sync
350+
* to ensure that the toast table gets fsync'd too.
351+
*
352+
* It's obvious that we must do this when not WAL-logging. It's less
353+
* obvious that we have to do it even if we did WAL-log the pages. The
346354
* reason is the same as in tablecmds.c's copy_relation_data(): we're
347355
* writing data that's not in shared buffers, and so a CHECKPOINT
348356
* occurring during the rewriteheap operation won't have fsync'd data we
349357
* wrote before the checkpoint.
350358
*/
351359
if (RelationNeedsWAL(state->rs_new_rel))
352-
smgrimmedsync(state->rs_new_rel->rd_smgr,MAIN_FORKNUM);
360+
heap_sync(state->rs_new_rel);
353361

354362
logical_end_heap_rewrite(state);
355363

@@ -647,6 +655,9 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
647655
{
648656
intoptions=HEAP_INSERT_SKIP_FSM;
649657

658+
if (!state->rs_use_wal)
659+
options |=HEAP_INSERT_SKIP_WAL;
660+
650661
/*
651662
* While rewriting the heap for VACUUM FULL / CLUSTER, make sure data
652663
* for the TOAST table are not logically decoded. The main heap is
@@ -685,7 +696,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
685696
/* Doesn't fit, so write out the existing page */
686697

687698
/* XLOG stuff */
688-
if (RelationNeedsWAL(state->rs_new_rel))
699+
if (state->rs_use_wal)
689700
log_newpage(&state->rs_new_rel->rd_node,
690701
MAIN_FORKNUM,
691702
state->rs_blockno,

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp