Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commita196073

Browse files
committed
[Issue#174] archive-push improvements: batching, multi-thread support, checksum computation on remote agent, use O_EXCL flag, --no-sync flag support
1 parent943548c commita196073

14 files changed

+1239
-344
lines changed

‎Makefile

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ OBJS += src/pg_crc.o src/datapagemap.o src/receivelog.o src/streamutil.o \
1515

1616
EXTRA_CLEAN = src/pg_crc.c src/datapagemap.c src/datapagemap.h\
1717
src/receivelog.c src/receivelog.h src/streamutil.c src/streamutil.h\
18-
src/xlogreader.c
18+
src/xlogreader.c src/instr_time.h
1919

20-
INCLUDES = src/datapagemap.h src/streamutil.h src/receivelog.h
20+
INCLUDES = src/datapagemap.h src/streamutil.h src/receivelog.h src/instr_time.h
2121

2222
ifdefUSE_PGXS
2323
PG_CONFIG = pg_config
@@ -60,6 +60,8 @@ all: checksrcdir $(INCLUDES);
6060

6161
$(PROGRAM):$(OBJS)
6262

63+
src/instr_time.h:$(top_srcdir)/src/include/portability/instr_time.h
64+
rm -f$@&&$(LN_S)$(srchome)/src/include/portability/instr_time.h$@
6365
src/datapagemap.c:$(top_srcdir)/src/bin/pg_rewind/datapagemap.c
6466
rm -f$@&&$(LN_S)$(srchome)/src/bin/pg_rewind/datapagemap.c$@
6567
src/datapagemap.h:$(top_srcdir)/src/bin/pg_rewind/datapagemap.h

‎doc/pgprobackup.xml

Lines changed: 69 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,6 @@ doc/src/sgml/pgprobackup.sgml
131131
<argchoice="plain"><option>archive-push</option></arg>
132132
<argchoice="plain"><option>-B</option> <replaceable>backup_dir</replaceable></arg>
133133
<argchoice="plain"><option>--instance</option> <replaceable>instance_name</replaceable></arg>
134-
<argchoice="plain"><option>--wal-file-path</option> <replaceable>wal_file_path</replaceable></arg>
135134
<argchoice="plain"><option>--wal-file-name</option> <replaceable>wal_file_name</replaceable></arg>
136135
<argrep="repeat"><replaceable>option</replaceable></arg>
137136
</cmdsynopsis>
@@ -786,7 +785,7 @@ ALTER ROLE backup WITH REPLICATION;
786785
parameter, as follows:
787786
</para>
788787
<programlisting>
789-
archive_command = '<replaceable>install_dir</replaceable>/pg_probackup archive-push -B <replaceable>backup_dir</replaceable> --instance <replaceable>instance_name</replaceable> --wal-file-path=%p --wal-file-name=%f [<replaceable>remote_options</replaceable>]'
788+
archive_command = '<replaceable>install_dir</replaceable>/pg_probackup archive-push -B <replaceable>backup_dir</replaceable> --instance <replaceable>instance_name</replaceable> --wal-file-name=%f [<replaceable>remote_options</replaceable>]'
790789
</programlisting>
791790
</listitem>
792791
</itemizedlist>
@@ -3934,9 +3933,12 @@ pg_probackup delete -B <replaceable>backup_dir</replaceable> --instance <replace
39343933
<title>archive-push</title>
39353934
<programlisting>
39363935
pg_probackup archive-push -B <replaceable>backup_dir</replaceable> --instance <replaceable>instance_name</replaceable>
3937-
--wal-file-path=<replaceable>wal_file_path</replaceable> --wal-file-name=<replaceable>wal_file_name</replaceable>
3938-
[--help] [--compress] [--compress-algorithm=<replaceable>compression_algorithm</replaceable>]
3939-
[--compress-level=<replaceable>compression_level</replaceable>] [--overwrite]
3936+
--wal-file-name=<replaceable>wal_file_name</replaceable>
3937+
[--help] [--no-sync] [--compress] [--no-ready-rename] [--overwrite]
3938+
[-j <replaceable>num_threads</replaceable>] [--batch-size=<replaceable>batch_size</replaceable>]
3939+
[--archive-timeout=<replaceable>timeout</replaceable>]
3940+
[--compress-algorithm=<replaceable>compression_algorithm</replaceable>]
3941+
[--compress-level=<replaceable>compression_level</replaceable>]
39403942
[<replaceable>remote_options</replaceable>] [<replaceable>logging_options</replaceable>]
39413943
</programlisting>
39423944
<para>
@@ -3961,13 +3963,24 @@ pg_probackup archive-push -B <replaceable>backup_dir</replaceable> --instance <r
39613963
with the <option>--overwrite</option> flag.
39623964
</para>
39633965
<para>
3964-
The files are copied to a temporary file with the
3965-
<literal>.part</literal> suffix. After the copy is
3966-
done, atomic rename is performed. This algorithm ensures that a
3967-
failed <command>archive-push</command> will not stall continuous archiving and
3968-
that concurrent archiving from multiple sources into a single
3969-
WAL archive have no risk of archive corruption. WAL segments copied to
3970-
the archive are synced to disk.
3966+
Every file is copied to a temporary file with the
3967+
<literal>.part</literal> suffix. If the temporary file already
3968+
exists, <application>pg_probackup</application> will wait
3969+
<option>archive_timeout</option> seconds before discarding it.
3970+
After the copy is done, atomic rename is performed.
3971+
This algorithm ensures that a failed <command>archive-push</command>
3972+
will not stall continuous archiving and that concurrent archiving from
3973+
multiple sources into a single WAL archive have no risk of archive
3974+
corruption.
3975+
</para>
3976+
<para>
3977+
To speed up archiving, especially in remote mode, <command>archive-push</command>
3978+
can be run on multiple threads using <option>-j num_threads</option> option.
3979+
Files can also be copied in batches using option <option>--batch-size</option>.
3980+
</para>
3981+
<para>
3982+
WAL segments copied to the archive are synced to disk unless
3983+
<option>--no-sync</option> flag is used.
39713984
</para>
39723985
<para>
39733986
You can use <command>archive-push</command> in the
@@ -4073,7 +4086,8 @@ pg_probackup archive-get -B <replaceable>backup_dir</replaceable> --instance <re
40734086
<para>
40744087
Sets the number of parallel threads for <command>backup</command>,
40754088
<command>restore</command>, <command>merge</command>,
4076-
<command>validate</command>, and <command>checkdb</command> processes.
4089+
<command>validate</command>, <command>checkdb</command> and
4090+
<command>archive-push</command> processes.
40774091
</para>
40784092
</listitem>
40794093
</varlistentry>
@@ -4732,6 +4746,48 @@ pg_probackup archive-get -B <replaceable>backup_dir</replaceable> --instance <re
47324746
</para>
47334747
</listitem>
47344748
</varlistentry>
4749+
4750+
<varlistentry>
4751+
<term><option>--batch-size=batch_size</option></term>
4752+
<listitem>
4753+
<para>
4754+
Sets the maximum number of files to be copied into archive by signle
4755+
<command>archive-push</command> process.
4756+
</para>
4757+
</listitem>
4758+
</varlistentry>
4759+
4760+
<varlistentry>
4761+
<term><option>--archive-timeout=<replaceable>wait_time</replaceable></option></term>
4762+
<listitem>
4763+
<para>
4764+
Sets the timeout for considering existing <literal>.part</literal> file to be stale. By default <application>pg_probackup</application> waits 300 seconds.
4765+
</para>
4766+
</listitem>
4767+
</varlistentry>
4768+
4769+
<varlistentry>
4770+
<term><option>--no-ready-rename</option></term>
4771+
<listitem>
4772+
<para>
4773+
Do not rename status files in <literal>archive_status</literal> directory.
4774+
This option should be used only if <parameter>archive_command</parameter>
4775+
contain multiple commands.
4776+
</para>
4777+
</listitem>
4778+
</varlistentry>
4779+
4780+
<varlistentry>
4781+
<term><option>--no-sync</option></term>
4782+
<listitem>
4783+
<para>
4784+
Do not sync copied WAL files to disk. You can use this flag to speed
4785+
up archiving process. Using this flag can result in WAL archive
4786+
corruption in case of operating system or hardware crash.
4787+
</para>
4788+
</listitem>
4789+
</varlistentry>
4790+
47354791
</variablelist>
47364792
</para>
47374793
</refsect3>

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp