Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit0ad8032

Browse files
committed
Server-side gzip compression.
pg_basebackup's --compression option now lets you write either"client-gzip" or "server-gzip" instead of just "gzip" to specifywhere the compression should be performed. If you write simply"gzip" it's taken to mean "client-gzip" unless you also use--target, in which case it is interpreted to mean "server-gzip",because that's the only thing that makes any sense in that case.To make this work, the BASE_BACKUP command now takes newCOMPRESSION and COMPRESSION_LEVEL options.At present, pg_basebackup cannot decompress .gz files, soserver-side compression will cause a failure if (1) -Ft is notused or (2) -R is used or (3) -D- is used without --no-manifest.Along the way, I removed the information message added by commit5c649fe which occurred if youspecified no compression level and told you that the default levelhad been used instead. That seemed like more output than mostpeople would want.Also along the way, this adds a check to the server forunrecognized base backup options. This repairs a bug introducedby commit0ba281c.This commit also adds some new test cases for pg_verifybackup.They take a server-side backup with and without compression, andthen extract the backup if we have the OS facilities availableto do so, and then run pg_verifybackup on the extracteddirectory. That is a good test of the functionality added bythis commit and also improves test coverage for the backup targetpatch (commit3500ccc) and forpg_verifybackup itself.Patch by me, with a bug fix by Jeevan Ladhe. The patch set of whichthis is a part has also had review and/or testing from Tushar Ahuja,Suraj Kharage, Dipesh Pandit, and Mark Dilger.Discussion:http://postgr.es/m/CA+Tgmoa-ST7fMLsVJduOB7Eub=2WjfpHS+QxHVEpUoinf4bOSg@mail.gmail.com
1 parentaa01051 commit0ad8032

File tree

10 files changed

+641
-24
lines changed

10 files changed

+641
-24
lines changed

‎doc/src/sgml/protocol.sgml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2719,6 +2719,28 @@ The commands accepted in replication mode are:
27192719
</listitem>
27202720
</varlistentry>
27212721

2722+
<varlistentry>
2723+
<term><literal>COMPRESSION</literal> <replaceable>'method'</replaceable></term>
2724+
<listitem>
2725+
<para>
2726+
Instructs the server to compress the backup using the specified
2727+
method. Currently, the only supported method is
2728+
<literal>gzip</literal>.
2729+
</para>
2730+
</listitem>
2731+
</varlistentry>
2732+
2733+
<varlistentry>
2734+
<term><literal>COMPRESSION_LEVEL</literal> <replaceable>level</replaceable></term>
2735+
<listitem>
2736+
<para>
2737+
Specifies the compression level to be used. This should only be
2738+
used in conjunction with the <literal>COMPRESSION</literal> option.
2739+
The value should be an integer between 1 and 9.
2740+
</para>
2741+
</listitem>
2742+
</varlistentry>
2743+
27222744
<varlistentry>
27232745
<term><literal>MAX_RATE</literal> <replaceable>rate</replaceable></term>
27242746
<listitem>

‎doc/src/sgml/ref/pg_basebackup.sgml

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -400,21 +400,36 @@ PostgreSQL documentation
400400
<term><option>-Z <replaceable class="parameter">level</replaceable></option></term>
401401
<term><option>-Z <replaceable class="parameter">method</replaceable></option>[:<replaceable>level</replaceable>]</term>
402402
<term><option>--compress=<replaceable class="parameter">level</replaceable></option></term>
403-
<term><option>--compress=<replaceable class="parameter">method</replaceable></option>[:<replaceable>level</replaceable>]</term>
403+
<term><option>--compress=[[{<replaceable class="parameter">client|server</replaceable>-}]<replaceable class="parameter">method</replaceable></option>[:<replaceable>level</replaceable>]</term>
404404
<listitem>
405405
<para>
406-
Enables compression of tar file output, and specifies the
407-
compression level (0 through 9, 0 being no compression and 9 being best
408-
compression). Compression is only available when using the tar
409-
format, and the suffix <filename>.gz</filename> will
410-
automatically be added to all tar filenames.
406+
Requests compression of the backup. If <literal>client</literal> or
407+
<literal>server</literal> is included, it specifies where the
408+
compression is to be performed. Compressing on the server will reduce
409+
transfer bandwidth but will increase server CPU consumption. The
410+
default is <literal>client</literal> except when
411+
<literal>--target</literal> is used. In that case, the backup is not
412+
being sent to the client, so only server compression is sensible.
413+
When <literal>-Xstream</literal>, which is the default, is used,
414+
server-side compression will not be applied to the WAL. To compress
415+
the WAL, use client-side compression, or
416+
specify <literal>-Xfetch</literal>.
411417
</para>
412418
<para>
413419
The compression method can be set to either <literal>gzip</literal>
414420
for compression with <application>gzip</application>, or
415421
<literal>none</literal> for no compression. A compression level
416422
can be optionally specified, by appending the level number after a
417-
colon (<literal>:</literal>).
423+
colon (<literal>:</literal>). If no level is specified, the default
424+
compression level will be used. If only a level is specified without
425+
mentioning an algorithm, <literal>gzip</literal> compression will
426+
be used if the level is greater than 0, and no compression will be
427+
used if the level is 0.
428+
</para>
429+
<para>
430+
When the tar format is used, the suffix <filename>.gz</filename> will
431+
automatically be added to all tar filenames. Compression is not
432+
available in plain format.
418433
</para>
419434
</listitem>
420435
</varlistentry>

‎src/backend/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ OBJS = \
4848
LIBS :=$(filter-out -lpgport -lpgcommon,$(LIBS))$(LDAP_LIBS_BE)$(ICU_LIBS)
4949

5050
# The backend doesn't need everything that's in LIBS, however
51-
LIBS :=$(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses,$(LIBS))
51+
LIBS :=$(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses,$(LIBS))
5252

5353
ifeq ($(with_systemd),yes)
5454
LIBS += -lsystemd

‎src/backend/replication/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ OBJS = \
1818
backup_manifest.o\
1919
basebackup.o\
2020
basebackup_copy.o\
21+
basebackup_gzip.o\
2122
basebackup_progress.o\
2223
basebackup_server.o\
2324
basebackup_sink.o\

‎src/backend/replication/basebackup.c

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,12 @@ typedef enum
6161
BACKUP_TARGET_SERVER
6262
}backup_target_type;
6363

64+
typedefenum
65+
{
66+
BACKUP_COMPRESSION_NONE,
67+
BACKUP_COMPRESSION_GZIP
68+
}basebackup_compression_type;
69+
6470
typedefstruct
6571
{
6672
constchar*label;
@@ -73,6 +79,8 @@ typedef struct
7379
backup_target_typetarget;
7480
char*target_detail;
7581
backup_manifest_optionmanifest;
82+
basebackup_compression_typecompression;
83+
intcompression_level;
7684
pg_checksum_typemanifest_checksum_type;
7785
}basebackup_options;
7886

@@ -707,11 +715,14 @@ parse_basebackup_options(List *options, basebackup_options *opt)
707715
boolo_target= false;
708716
boolo_target_detail= false;
709717
char*target_str="compat";/* placate compiler */
718+
boolo_compression= false;
719+
boolo_compression_level= false;
710720

711721
MemSet(opt,0,sizeof(*opt));
712722
opt->target=BACKUP_TARGET_COMPAT;
713723
opt->manifest=MANIFEST_OPTION_NO;
714724
opt->manifest_checksum_type=CHECKSUM_TYPE_CRC32C;
725+
opt->compression=BACKUP_COMPRESSION_NONE;
715726

716727
foreach(lopt,options)
717728
{
@@ -881,7 +892,41 @@ parse_basebackup_options(List *options, basebackup_options *opt)
881892
opt->target_detail=optval;
882893
o_target_detail= true;
883894
}
895+
elseif (strcmp(defel->defname,"compression")==0)
896+
{
897+
char*optval=defGetString(defel);
898+
899+
if (o_compression)
900+
ereport(ERROR,
901+
(errcode(ERRCODE_SYNTAX_ERROR),
902+
errmsg("duplicate option \"%s\"",defel->defname)));
903+
if (strcmp(optval,"none")==0)
904+
opt->compression=BACKUP_COMPRESSION_NONE;
905+
elseif (strcmp(optval,"gzip")==0)
906+
opt->compression=BACKUP_COMPRESSION_GZIP;
907+
else
908+
ereport(ERROR,
909+
(errcode(ERRCODE_SYNTAX_ERROR),
910+
errmsg("unrecognized compression algorithm: \"%s\"",
911+
optval)));
912+
o_compression= true;
913+
}
914+
elseif (strcmp(defel->defname,"compression_level")==0)
915+
{
916+
if (o_compression_level)
917+
ereport(ERROR,
918+
(errcode(ERRCODE_SYNTAX_ERROR),
919+
errmsg("duplicate option \"%s\"",defel->defname)));
920+
opt->compression_level=defGetInt32(defel);
921+
o_compression_level= true;
922+
}
923+
else
924+
ereport(ERROR,
925+
(errcode(ERRCODE_SYNTAX_ERROR),
926+
errmsg("unrecognized base backup option: \"%s\"",
927+
defel->defname)));
884928
}
929+
885930
if (opt->label==NULL)
886931
opt->label="base backup";
887932
if (opt->manifest==MANIFEST_OPTION_NO)
@@ -908,6 +953,11 @@ parse_basebackup_options(List *options, basebackup_options *opt)
908953
errmsg("target '%s' does not accept a target detail",
909954
target_str)));
910955
}
956+
957+
if (o_compression_level&& !o_compression)
958+
ereport(ERROR,
959+
(errcode(ERRCODE_SYNTAX_ERROR),
960+
errmsg("compression level requires compression")));
911961
}
912962

913963

@@ -975,6 +1025,10 @@ SendBaseBackup(BaseBackupCmd *cmd)
9751025
if (opt.maxrate>0)
9761026
sink=bbsink_throttle_new(sink,opt.maxrate);
9771027

1028+
/* Set up server-side compression, if client requested it */
1029+
if (opt.compression==BACKUP_COMPRESSION_GZIP)
1030+
sink=bbsink_gzip_new(sink,opt.compression_level);
1031+
9781032
/* Set up progress reporting. */
9791033
sink=bbsink_progress_new(sink,opt.progress);
9801034

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp