Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commited308d7

Browse files
committed
Add options to enable and disable checksums in pg_checksums
An offline cluster can now work with more modes in pg_checksums:- --enable enables checksums in a cluster, updating all blocks with acorrect checksum, and updating the control file at the end.- --disable disables checksums in a cluster, updating only the controlfile.- --check is an extra option able to verify checksums for a cluster, andthe default used if no mode is specified.When running --enable or --disable, the data folder gets fsync'd fordurability, and then it is followed by a control file update and flushto keep the operation consistent should the tool be interrupted, killedor the host unplugged. If no mode is specified in the options, then--check is used for compatibility with older versions of pg_checksums(named pg_verify_checksums in v11 where it was introduced).Author: Michael Banck, Michael PaquierReviewed-by: Fabien Coelho, Magnus Hagander, Sergei KornilovDiscussion:https://postgr.es/m/20181221201616.GD4974@nighthawk.caipicrew.dd-dns.de
1 parent87914e7 commited308d7

File tree

4 files changed

+285
-46
lines changed

4 files changed

+285
-46
lines changed

‎doc/src/sgml/ref/pg_checksums.sgml

Lines changed: 74 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ PostgreSQL documentation
1616

1717
<refnamediv>
1818
<refname>pg_checksums</refname>
19-
<refpurpose>verify data checksums in a <productname>PostgreSQL</productname> database cluster</refpurpose>
19+
<refpurpose>enable, disable or check data checksums in a <productname>PostgreSQL</productname> database cluster</refpurpose>
2020
</refnamediv>
2121

2222
<refsynopsisdiv>
@@ -36,10 +36,19 @@ PostgreSQL documentation
3636
<refsect1 id="r1-app-pg_checksums-1">
3737
<title>Description</title>
3838
<para>
39-
<application>pg_checksums</application> verifies data checksums in a
40-
<productname>PostgreSQL</productname> cluster. The server must be shut
41-
down cleanly before running <application>pg_checksums</application>.
42-
The exit status is zero if there are no checksum errors, otherwise nonzero.
39+
<application>pg_checksums</application> checks, enables or disables data
40+
checksums in a <productname>PostgreSQL</productname> cluster. The server
41+
must be shut down cleanly before running
42+
<application>pg_checksums</application>. The exit status is zero if there
43+
are no checksum errors when checking them, and nonzero if at least one
44+
checksum failure is detected. If enabling or disabling checksums, the
45+
exit status is nonzero if the operation failed.
46+
</para>
47+
48+
<para>
49+
While checking or enabling checksums needs to scan or write every file in
50+
the cluster, disabling checksums will only update the file
51+
<filename>pg_control</filename>.
4352
</para>
4453
</refsect1>
4554

@@ -60,6 +69,37 @@ PostgreSQL documentation
6069
</listitem>
6170
</varlistentry>
6271

72+
<varlistentry>
73+
<term><option>-c</option></term>
74+
<term><option>--check</option></term>
75+
<listitem>
76+
<para>
77+
Checks checksums. This is the default mode if nothing else is
78+
specified.
79+
</para>
80+
</listitem>
81+
</varlistentry>
82+
83+
<varlistentry>
84+
<term><option>-d</option></term>
85+
<term><option>--disable</option></term>
86+
<listitem>
87+
<para>
88+
Disables checksums.
89+
</para>
90+
</listitem>
91+
</varlistentry>
92+
93+
<varlistentry>
94+
<term><option>-e</option></term>
95+
<term><option>--enable</option></term>
96+
<listitem>
97+
<para>
98+
Enables checksums.
99+
</para>
100+
</listitem>
101+
</varlistentry>
102+
63103
<varlistentry>
64104
<term><option>-v</option></term>
65105
<term><option>--verbose</option></term>
@@ -119,4 +159,33 @@ PostgreSQL documentation
119159
</varlistentry>
120160
</variablelist>
121161
</refsect1>
162+
163+
<refsect1>
164+
<title>Notes</title>
165+
<para>
166+
When disabling or enabling checksums in a replication setup of multiple
167+
clusters, it is recommended to stop all the clusters before doing
168+
the switch to all the clusters consistently. When using a replication
169+
setup with tools which perform direct copies of relation file blocks
170+
(for example <xref linkend="app-pgrewind"/>), enabling or disabling
171+
checksums can lead to page corruptions in the shape of incorrect
172+
checksums if the operation is not done consistently across all nodes.
173+
Destroying all the standbys in the setup first, enabling or disabling
174+
checksums on the primary and finally recreating the standbys from
175+
scratch is also safe.
176+
</para>
177+
<para>
178+
If <application>pg_checksums</application> is aborted or killed in
179+
its operation while enabling or disabling checksums, the cluster
180+
will have the same state with respect of checksums as before the
181+
operation and <application>pg_checksums</application> needs to be
182+
restarted.
183+
</para>
184+
<para>
185+
When enabling checksums in a cluster, the operation can potentially
186+
take a long time if the data directory is large. During this operation,
187+
the cluster or other programs that write to the data directory must not
188+
be started or else data loss may occur.
189+
</para>
190+
</refsect1>
122191
</refentry>

‎src/bin/pg_checksums/pg_checksums.c

Lines changed: 149 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
/*-------------------------------------------------------------------------
22
*
33
* pg_checksums.c
4-
* Verifies page level checksums in an offline cluster.
4+
* Checks, enables or disables page level checksums for an offline
5+
* cluster
56
*
67
* Copyright (c) 2010-2019, PostgreSQL Global Development Group
78
*
@@ -17,14 +18,15 @@
1718
#include<sys/stat.h>
1819
#include<unistd.h>
1920

20-
#include"catalog/pg_control.h"
21+
#include"access/xlog_internal.h"
2122
#include"common/controldata_utils.h"
23+
#include"common/file_perm.h"
24+
#include"common/file_utils.h"
2225
#include"getopt_long.h"
2326
#include"pg_getopt.h"
2427
#include"storage/bufpage.h"
2528
#include"storage/checksum.h"
2629
#include"storage/checksum_impl.h"
27-
#include"storage/fd.h"
2830

2931

3032
staticint64files=0;
@@ -35,16 +37,38 @@ static ControlFileData *ControlFile;
3537
staticchar*only_relfilenode=NULL;
3638
staticboolverbose= false;
3739

40+
typedefenum
41+
{
42+
PG_MODE_CHECK,
43+
PG_MODE_DISABLE,
44+
PG_MODE_ENABLE
45+
}PgChecksumMode;
46+
47+
/*
48+
* Filename components.
49+
*
50+
* XXX: fd.h is not declared here as frontend side code is not able to
51+
* interact with the backend-side definitions for the various fsync
52+
* wrappers.
53+
*/
54+
#definePG_TEMP_FILES_DIR "pgsql_tmp"
55+
#definePG_TEMP_FILE_PREFIX "pgsql_tmp"
56+
57+
staticPgChecksumModemode=PG_MODE_CHECK;
58+
3859
staticconstchar*progname;
3960

4061
staticvoid
4162
usage(void)
4263
{
43-
printf(_("%s verifies data checksums in a PostgreSQL database cluster.\n\n"),progname);
64+
printf(_("%senables, disables orverifies data checksums in a PostgreSQL database cluster.\n\n"),progname);
4465
printf(_("Usage:\n"));
4566
printf(_(" %s [OPTION]... [DATADIR]\n"),progname);
4667
printf(_("\nOptions:\n"));
4768
printf(_(" [-D, --pgdata=]DATADIR data directory\n"));
69+
printf(_(" -c, --check check data checksums (default)\n"));
70+
printf(_(" -d, --disable disable data checksums\n"));
71+
printf(_(" -e, --enable enable data checksums\n"));
4872
printf(_(" -v, --verbose output verbose messages\n"));
4973
printf(_(" -r RELFILENODE check only relation with specified relfilenode\n"));
5074
printf(_(" -V, --version output version information, then exit\n"));
@@ -90,8 +114,14 @@ scan_file(const char *fn, BlockNumber segmentno)
90114
PageHeaderheader= (PageHeader)buf.data;
91115
intf;
92116
BlockNumberblockno;
117+
intflags;
118+
119+
Assert(mode==PG_MODE_ENABLE||
120+
mode==PG_MODE_CHECK);
121+
122+
flags= (mode==PG_MODE_ENABLE) ?O_RDWR :O_RDONLY;
123+
f=open(fn,PG_BINARY |flags,0);
93124

94-
f=open(fn,O_RDONLY |PG_BINARY,0);
95125
if (f<0)
96126
{
97127
fprintf(stderr,_("%s: could not open file \"%s\": %s\n"),
@@ -121,18 +151,47 @@ scan_file(const char *fn, BlockNumber segmentno)
121151
continue;
122152

123153
csum=pg_checksum_page(buf.data,blockno+segmentno*RELSEG_SIZE);
124-
if (csum!=header->pd_checksum)
154+
if (mode==PG_MODE_CHECK)
125155
{
126-
if (ControlFile->data_checksum_version==PG_DATA_CHECKSUM_VERSION)
127-
fprintf(stderr,_("%s: checksum verification failed in file \"%s\", block %u: calculated checksum %X but block contains %X\n"),
128-
progname,fn,blockno,csum,header->pd_checksum);
129-
badblocks++;
156+
if (csum!=header->pd_checksum)
157+
{
158+
if (ControlFile->data_checksum_version==PG_DATA_CHECKSUM_VERSION)
159+
fprintf(stderr,_("%s: checksum verification failed in file \"%s\", block %u: calculated checksum %X but block contains %X\n"),
160+
progname,fn,blockno,csum,header->pd_checksum);
161+
badblocks++;
162+
}
163+
}
164+
elseif (mode==PG_MODE_ENABLE)
165+
{
166+
/* Set checksum in page header */
167+
header->pd_checksum=csum;
168+
169+
/* Seek back to beginning of block */
170+
if (lseek(f,-BLCKSZ,SEEK_CUR)<0)
171+
{
172+
fprintf(stderr,_("%s: seek failed for block %d in file \"%s\": %s\n"),progname,blockno,fn,strerror(errno));
173+
exit(1);
174+
}
175+
176+
/* Write block with checksum */
177+
if (write(f,buf.data,BLCKSZ)!=BLCKSZ)
178+
{
179+
fprintf(stderr,_("%s: could not update checksum of block %d in file \"%s\": %s\n"),
180+
progname,blockno,fn,strerror(errno));
181+
exit(1);
182+
}
130183
}
131184
}
132185

133186
if (verbose)
134-
fprintf(stderr,
135-
_("%s: checksums verified in file \"%s\"\n"),progname,fn);
187+
{
188+
if (mode==PG_MODE_CHECK)
189+
fprintf(stderr,
190+
_("%s: checksums verified in file \"%s\"\n"),progname,fn);
191+
if (mode==PG_MODE_ENABLE)
192+
fprintf(stderr,
193+
_("%s: checksums enabled in file \"%s\"\n"),progname,fn);
194+
}
136195

137196
close(f);
138197
}
@@ -234,7 +293,10 @@ int
234293
main(intargc,char*argv[])
235294
{
236295
staticstructoptionlong_options[]= {
296+
{"check",no_argument,NULL,'c'},
237297
{"pgdata",required_argument,NULL,'D'},
298+
{"disable",no_argument,NULL,'d'},
299+
{"enable",no_argument,NULL,'e'},
238300
{"verbose",no_argument,NULL,'v'},
239301
{NULL,0,NULL,0}
240302
};
@@ -262,10 +324,19 @@ main(int argc, char *argv[])
262324
}
263325
}
264326

265-
while ((c=getopt_long(argc,argv,"D:r:v",long_options,&option_index))!=-1)
327+
while ((c=getopt_long(argc,argv,"cD:der:v",long_options,&option_index))!=-1)
266328
{
267329
switch (c)
268330
{
331+
case'c':
332+
mode=PG_MODE_CHECK;
333+
break;
334+
case'd':
335+
mode=PG_MODE_DISABLE;
336+
break;
337+
case'e':
338+
mode=PG_MODE_ENABLE;
339+
break;
269340
case'v':
270341
verbose= true;
271342
break;
@@ -312,6 +383,15 @@ main(int argc, char *argv[])
312383
exit(1);
313384
}
314385

386+
/* Relfilenode checking only works in --check mode */
387+
if (mode!=PG_MODE_CHECK&&only_relfilenode)
388+
{
389+
fprintf(stderr,_("%s: relfilenode option only possible with --check\n"),progname);
390+
fprintf(stderr,_("Try \"%s --help\" for more information.\n"),
391+
progname);
392+
exit(1);
393+
}
394+
315395
/* Check if cluster is running */
316396
ControlFile=get_controlfile(DataDir,progname,&crc_ok);
317397
if (!crc_ok)
@@ -339,29 +419,72 @@ main(int argc, char *argv[])
339419
if (ControlFile->state!=DB_SHUTDOWNED&&
340420
ControlFile->state!=DB_SHUTDOWNED_IN_RECOVERY)
341421
{
342-
fprintf(stderr,_("%s: cluster must be shut down to verify checksums\n"),progname);
422+
fprintf(stderr,_("%s: cluster must be shut down\n"),progname);
343423
exit(1);
344424
}
345425

346-
if (ControlFile->data_checksum_version==0)
426+
if (ControlFile->data_checksum_version==0&&
427+
mode==PG_MODE_CHECK)
347428
{
348429
fprintf(stderr,_("%s: data checksums are not enabled in cluster\n"),progname);
349430
exit(1);
350431
}
432+
if (ControlFile->data_checksum_version==0&&
433+
mode==PG_MODE_DISABLE)
434+
{
435+
fprintf(stderr,_("%s: data checksums are already disabled in cluster.\n"),progname);
436+
exit(1);
437+
}
438+
if (ControlFile->data_checksum_version>0&&
439+
mode==PG_MODE_ENABLE)
440+
{
441+
fprintf(stderr,_("%s: data checksums are already enabled in cluster.\n"),progname);
442+
exit(1);
443+
}
444+
445+
/* Operate on all files if checking or enabling checksums */
446+
if (mode==PG_MODE_CHECK||mode==PG_MODE_ENABLE)
447+
{
448+
scan_directory(DataDir,"global");
449+
scan_directory(DataDir,"base");
450+
scan_directory(DataDir,"pg_tblspc");
451+
452+
printf(_("Checksum operation completed\n"));
453+
printf(_("Files scanned: %s\n"),psprintf(INT64_FORMAT,files));
454+
printf(_("Blocks scanned: %s\n"),psprintf(INT64_FORMAT,blocks));
455+
if (mode==PG_MODE_CHECK)
456+
{
457+
printf(_("Bad checksums: %s\n"),psprintf(INT64_FORMAT,badblocks));
458+
printf(_("Data checksum version: %d\n"),ControlFile->data_checksum_version);
459+
460+
if (badblocks>0)
461+
exit(1);
462+
}
463+
}
464+
465+
/*
466+
* Finally make the data durable on disk if enabling or disabling
467+
* checksums. Flush first the data directory for safety, and then update
468+
* the control file to keep the switch consistent.
469+
*/
470+
if (mode==PG_MODE_ENABLE||mode==PG_MODE_DISABLE)
471+
{
472+
ControlFile->data_checksum_version=
473+
(mode==PG_MODE_ENABLE) ?PG_DATA_CHECKSUM_VERSION :0;
351474

352-
/* Scan all files */
353-
scan_directory(DataDir,"global");
354-
scan_directory(DataDir,"base");
355-
scan_directory(DataDir,"pg_tblspc");
475+
printf(_("Syncing data directory\n"));
476+
fsync_pgdata(DataDir,progname,PG_VERSION_NUM);
356477

357-
printf(_("Checksum scan completed\n"));
358-
printf(_("Data checksum version: %d\n"),ControlFile->data_checksum_version);
359-
printf(_("Files scanned: %s\n"),psprintf(INT64_FORMAT,files));
360-
printf(_("Blocks scanned: %s\n"),psprintf(INT64_FORMAT,blocks));
361-
printf(_("Bad checksums: %s\n"),psprintf(INT64_FORMAT,badblocks));
478+
printf(_("Updating control file\n"));
479+
update_controlfile(DataDir,progname,ControlFile, true);
362480

363-
if (badblocks>0)
364-
return1;
481+
if (verbose)
482+
printf(_("Data checksum version: %d\n"),ControlFile->data_checksum_version);
483+
if (mode==PG_MODE_ENABLE)
484+
printf(_("Checksums enabled in cluster\n"));
485+
else
486+
printf(_("Checksums disabled in cluster\n"));
487+
}
365488

366489
return0;
367490
}

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp