Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitffc9dda

Browse files
committed
Add TAP tests for ZLIB compression for pg_receivewal
There is a non-trivial amount of code that handles ZLIB compression inpg_receivewal, from basics like the format name, the calculation of thestart streaming position and of course the compression itself, but therewas no automated coverage for it.This commit introduces a set of conditional tests (if the build supportsZLIB) to cover the creation of ZLIB-compressed WAL segments, thehandling of the partial, compressed, WAL segments and the compressionoperation in itself. Note that there is an extra phase checking thevalidity of the generated files by using directly a gzip command, passeddown by the Makefile of pg_receivewal. This part is skipped if thecommand cannot be found, something likely going to happen on Windowswith MSVC except if one sets the variable GZIP_PROGRAM in theenvironment of the test.This set of tests will become handy for upcoming patches that add moreoptions for the compression methods used by pg_receivewal, like LZ4, tomake sure that no existing facilities are broken.Author: Georgios KokolatosReviewed-by: Gilles Darold, Michael PaquierDiscussion:https://postgr.es/m/07BK3Mk5aEOsTwGaY77qBVyf9GjoEzn8TMgHLyPGfEFPIpTEmoQuP2P4c7teesjSg-LPeUafsp1flnPeQYINMSMB_UpggJDoduB5EDYBqaQ=@protonmail.com
1 parentdc2db1e commitffc9dda

File tree

2 files changed

+82
-3
lines changed

2 files changed

+82
-3
lines changed

‎src/bin/pg_basebackup/Makefile

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,12 @@ subdir = src/bin/pg_basebackup
1818
top_builddir = ../../..
1919
include$(top_builddir)/src/Makefile.global
2020

21-
# makethis available to TAP test scripts
21+
# makethese available to TAP test scripts
2222
exportTAR
23+
# Note that GZIP cannot be used directly as this environment variable is
24+
# used by the command "gzip" to pass down options, so stick with a different
25+
# name.
26+
exportGZIP_PROGRAM=$(GZIP)
2327

2428
overrideCPPFLAGS := -I$(libpq_srcdir)$(CPPFLAGS)
2529
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils$(libpq_pgport)

‎src/bin/pg_basebackup/t/020_pg_receivewal.pl

Lines changed: 77 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
use warnings;
66
use TestLib;
77
use PostgresNode;
8-
use Test::Moretests=>19;
8+
use Test::Moretests=>27;
99

1010
program_help_ok('pg_receivewal');
1111
program_version_ok('pg_receivewal');
@@ -58,14 +58,89 @@
5858
$primary->psql('postgres',
5959
'INSERT INTO test_table VALUES (generate_series(1,100));');
6060

61-
# Stream up to the given position.
61+
# Stream up to the given position. This is necessary to have a fixed
62+
# started point for the next commands done in this test, with or without
63+
# compression involved.
6264
$primary->command_ok(
6365
[
6466
'pg_receivewal','-D',$stream_dir,'--verbose',
6567
'--endpos',$nextlsn,'--synchronous','--no-loop'
6668
],
6769
'streaming some WAL with --synchronous');
6870

71+
# Verify that one partial file was generated and keep track of it
72+
my@partial_wals =glob"$stream_dir/*\.partial";
73+
is(scalar(@partial_wals), 1,"one partial WAL segment was created");
74+
75+
# Check ZLIB compression if available.
76+
SKIP:
77+
{
78+
skip"postgres was not built with ZLIB support", 5
79+
if (!check_pg_config("#define HAVE_LIBZ 1"));
80+
81+
# Generate more WAL worth one completed, compressed, segment.
82+
$primary->psql('postgres','SELECT pg_switch_wal();');
83+
$nextlsn =
84+
$primary->safe_psql('postgres','SELECT pg_current_wal_insert_lsn();');
85+
chomp($nextlsn);
86+
$primary->psql('postgres',
87+
'INSERT INTO test_table VALUES (generate_series(100,200));');
88+
89+
$primary->command_ok(
90+
[
91+
'pg_receivewal','-D',$stream_dir,'--verbose',
92+
'--endpos',$nextlsn,'--compress','1'
93+
],
94+
"streaming some WAL using ZLIB compression");
95+
96+
# Verify that the stored files are generated with their expected
97+
# names.
98+
my@zlib_wals =glob"$stream_dir/*.gz";
99+
is(scalar(@zlib_wals), 1,
100+
"one WAL segment compressed with ZLIB was created");
101+
my@zlib_partial_wals =glob"$stream_dir/*.gz.partial";
102+
is(scalar(@zlib_partial_wals),
103+
1,"one partial WAL segment compressed with ZLIB was created");
104+
105+
# Verify that the start streaming position is computed correctly by
106+
# comparing it with the partial file generated previously. The name
107+
# of the previous partial, now-completed WAL segment is updated, keeping
108+
# its base number.
109+
$partial_wals[0] =~s/\.partial$/.gz/;
110+
is($zlib_wals[0] =~m/$partial_wals[0]/,
111+
1,"one partial WAL segment is now completed");
112+
# Update the list of partial wals with the current one.
113+
@partial_wals =@zlib_partial_wals;
114+
115+
# There is one complete and one partial file compressed with ZLIB.
116+
# Check the integrity of both, if gzip is a command available.
117+
my$gzip =$ENV{GZIP_PROGRAM};
118+
skip"program gzip is not found in your system", 1
119+
if ( !defined$gzip
120+
||$gzipeq''
121+
|| system_log($gzip,'--version') != 0);
122+
123+
push(@zlib_wals,@zlib_partial_wals);
124+
my$gzip_is_valid = system_log($gzip,'--test',@zlib_wals);
125+
is($gzip_is_valid, 0,
126+
"gzip verified the integrity of compressed WAL segments");
127+
}
128+
129+
# Verify that the start streaming position is computed and that the value is
130+
# correct regardless of whether ZLIB is available.
131+
$primary->psql('postgres','SELECT pg_switch_wal();');
132+
$nextlsn =
133+
$primary->safe_psql('postgres','SELECT pg_current_wal_insert_lsn();');
134+
chomp($nextlsn);
135+
$primary->psql('postgres',
136+
'INSERT INTO test_table VALUES (generate_series(200,300));');
137+
$primary->command_ok(
138+
['pg_receivewal','-D',$stream_dir,'--verbose','--endpos',$nextlsn ],
139+
"streaming some WAL");
140+
141+
$partial_wals[0] =~s/(\.gz)?.partial//;
142+
ok(-e$partial_wals[0],"check that previously partial WAL is now complete");
143+
69144
# Permissions on WAL files should be default
70145
SKIP:
71146
{

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp