Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitcec2edf

Browse files
author
Amit Kapila
committed
Add logical_decoding_work_mem to limit ReorderBuffer memory usage.
Instead of deciding to serialize a transaction merely based on thenumber of changes in that xact (toplevel or subxact), this makesthe decisions based on amount of memory consumed by the changes.The memory limit is defined by a new logical_decoding_work_mem GUC,so for example we can do this SET logical_decoding_work_mem = '128kB'to reduce the memory usage of walsenders or set the higher value toreduce disk writes. The minimum value is 64kB.When adding a change to a transaction, we account for the size intwo places. Firstly, in the ReorderBuffer, which is then used todecide if we reached the total memory limit. And secondly in thetransaction the change belongs to, so that we can pick the largesttransaction to evict (and serialize to disk).We still use max_changes_in_memory when loading changes serializedto disk. The trouble is we can't use the memory limit directly asthere might be multiple subxact serialized, we need to read all ofthem but we don't know how many are there (and which subxact toread first).We do not serialize the ReorderBufferTXN entries, so if there is atransaction with many subxacts, most memory may be in this type ofobjects. Those records are not included in the memory accounting.We also do not account for INTERNAL_TUPLECID changes, which arekept in a separate list and not evicted from memory. Transactionswith many CTID changes may consume significant amounts of memory,but we can't really do much about that.The current eviction algorithm is very simple - the transaction ispicked merely by size, while it might be useful to also consider age(LSN) of the changes for example. With the new Generational memoryallocator, evicting the oldest changes would make it more likelythe memory gets actually pfreed.The logical_decoding_work_mem can be set in postgresql.conf, in whichcase it serves as the default for all publishers on that instance.Author: Tomas Vondra, with changes by Dilip Kumar and Amit KapilaReviewed-by: Dilip Kumar and Amit KapilaTested-By: Vignesh CDiscussion:https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com
1 parent2110f71 commitcec2edf

File tree

6 files changed

+333
-12
lines changed

6 files changed

+333
-12
lines changed

‎contrib/test_decoding/logical.conf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
wal_level = logical
22
max_replication_slots = 4
3+
logical_decoding_work_mem = 64kB

‎doc/src/sgml/config.sgml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1732,6 +1732,27 @@ include_dir 'conf.d'
17321732
</listitem>
17331733
</varlistentry>
17341734

1735+
<varlistentry id="guc-logical-decoding-work-mem" xreflabel="logical_decoding_work_mem">
1736+
<term><varname>logical_decoding_work_mem</varname> (<type>integer</type>)
1737+
<indexterm>
1738+
<primary><varname>logical_decoding_work_mem</varname> configuration parameter</primary>
1739+
</indexterm>
1740+
</term>
1741+
<listitem>
1742+
<para>
1743+
Specifies the maximum amount of memory to be used by logical decoding,
1744+
before some of the decoded changes are written to local disk. This
1745+
limits the amount of memory used by logical streaming replication
1746+
connections. It defaults to 64 megabytes (<literal>64MB</literal>).
1747+
Since each replication connection only uses a single buffer of this size,
1748+
and an installation normally doesn't have many such connections
1749+
concurrently (as limited by <varname>max_wal_senders</varname>), it's
1750+
safe to set this value significantly higher than <varname>work_mem</varname>,
1751+
reducing the amount of decoded changes written to disk.
1752+
</para>
1753+
</listitem>
1754+
</varlistentry>
1755+
17351756
<varlistentry id="guc-max-stack-depth" xreflabel="max_stack_depth">
17361757
<term><varname>max_stack_depth</varname> (<type>integer</type>)
17371758
<indexterm>

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp