Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitfa146f5

Browse files
committed
Merge with master
2 parentsae5aa3d +5bcea32 commitfa146f5

File tree

52 files changed

+1791
-935
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+1791
-935
lines changed

‎README.md‎

Lines changed: 28 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,6 @@
55
The utility is compatible with:
66
* PostgreSQL 9.5, 9.6, 10, 11;
77

8-
`PTRACK` backup support provided via following options:
9-
* vanilla PostgreSQL compiled with ptrack patch. Currently there are patches for[PostgreSQL 9.6](https://gist.githubusercontent.com/gsmol/5b615c971dfd461c76ef41a118ff4d97/raw/e471251983f14e980041f43bea7709b8246f4178/ptrack_9.6.6_v1.5.patch) and[PostgreSQL 10](https://gist.githubusercontent.com/gsmol/be8ee2a132b88463821021fd910d960e/raw/de24f9499f4f314a4a3e5fae5ed4edb945964df8/ptrack_10.1_v1.5.patch)
10-
* Postgres Pro Standard 9.5, 9.6, 10, 11
11-
* Postgres Pro Enterprise 9.5, 9.6, 10
12-
138
As compared to other backup solutions,`pg_probackup` offers the following benefits that can help you implement different backup strategies and deal with large amounts of data:
149
* Choosing between full and page-level incremental backups to speed up backup and recovery
1510
* Implementing a single backup strategy for multi-server PostgreSQL clusters
@@ -34,12 +29,17 @@ Regardless of the chosen backup type, all backups taken with `pg_probackup` supp
3429
*`Autonomous backups` include all the files required to restore the cluster to a consistent state at the time the backup was taken. Even if continuous archiving is not set up, the required WAL segments are included into the backup.
3530
*`Archive backups` rely on continuous archiving. Such backups enable cluster recovery to an arbitrary point after the backup was taken (point-in-time recovery).
3631

32+
`PTRACK` backup support provided via following options:
33+
* vanilla PostgreSQL compiled with ptrack patch. Currently there are patches for[PostgreSQL 9.6](https://gist.githubusercontent.com/gsmol/5b615c971dfd461c76ef41a118ff4d97/raw/e471251983f14e980041f43bea7709b8246f4178/ptrack_9.6.6_v1.5.patch) and[PostgreSQL 10](https://gist.githubusercontent.com/gsmol/be8ee2a132b88463821021fd910d960e/raw/de24f9499f4f314a4a3e5fae5ed4edb945964df8/ptrack_10.1_v1.5.patch)
34+
* Postgres Pro Standard 9.5, 9.6, 10, 11
35+
* Postgres Pro Enterprise 9.5, 9.6, 10
36+
3737
##Limitations
3838

3939
`pg_probackup` currently has the following limitations:
4040
* Creating backups from a remote server is currently not supported.
4141
* The server from which the backup was taken and the restored server must be compatible by the[block_size](https://postgrespro.com/docs/postgresql/current/runtime-config-preset#GUC-BLOCK-SIZE) and[wal_block_size](https://postgrespro.com/docs/postgresql/current/runtime-config-preset#GUC-WAL-BLOCK-SIZE) parameters and have the same major release number.
42-
* Microsoft Windows operating system isnot supported.
42+
* Microsoft Windows operating systemsupportisin beta stage.
4343
* Configuration files outside of PostgreSQL data directory are not included into the backup and should be backed up separately.
4444

4545
##Installation and Setup
@@ -48,36 +48,51 @@ Regardless of the chosen backup type, all backups taken with `pg_probackup` supp
4848
#DEB Ubuntu|Debian Packages
4949
echo"deb [arch=amd64] http://repo.postgrespro.ru/pg_probackup/deb/$(lsb_release -cs) main-$(lsb_release -cs)"> /etc/apt/sources.list.d/pg_probackup.list
5050
wget -O - http://repo.postgrespro.ru/pg_probackup/keys/GPG-KEY-PG_PROBACKUP| apt-key add -&& apt-get update
51-
apt-get install pg-probackup-{10,9.6,9.5}
51+
apt-get install pg-probackup-{11,10,9.6,9.5}
5252

5353
#DEB-SRC Packages
5454
echo"deb-src [arch=amd64] http://repo.postgrespro.ru/pg_probackup/deb/$(lsb_release -cs) main-$(lsb_release -cs)">>\
5555
/etc/apt/sources.list.d/pg_probackup.list
56-
apt-getsource pg-probackup-{10,9.6,9.5}
56+
apt-getsource pg-probackup-{11,10,9.6,9.5}
5757

5858
#RPM Centos Packages
5959
rpm -ivh http://repo.postgrespro.ru/pg_probackup/keys/pg_probackup-repo-centos.noarch.rpm
60-
yum install pg_probackup-{10,9.6,9.5}
60+
yum install pg_probackup-{11,10,9.6,9.5}
6161

6262
#RPM RHEL Packages
6363
rpm -ivh http://repo.postgrespro.ru/pg_probackup/keys/pg_probackup-repo-rhel.noarch.rpm
64-
yum install pg_probackup-{10,9.6,9.5}
64+
yum install pg_probackup-{11,10,9.6,9.5}
6565

6666
#RPM Oracle Linux Packages
6767
rpm -ivh http://repo.postgrespro.ru/pg_probackup/keys/pg_probackup-repo-oraclelinux.noarch.rpm
68-
yum install pg_probackup-{10,9.6,9.5}
68+
yum install pg_probackup-{11,10,9.6,9.5}
6969

7070
#SRPM Packages
71-
yumdownloader --source pg_probackup-{10,9.6,9.5}
71+
yumdownloader --source pg_probackup-{11,10,9.6,9.5}
7272
```
7373

74+
Once you have`pg_probackup` installed, complete[the setup](https://postgrespro.com/docs/postgrespro/current/app-pgprobackup.html#pg-probackup-install-and-setup).
75+
76+
##Building from source
77+
###Linux
78+
7479
To compile`pg_probackup`, you must have a PostgreSQL installation and raw source tree. To install`pg_probackup`, execute this in the module's directory:
7580

7681
```shell
7782
make USE_PGXS=1 PG_CONFIG=<path_to_pg_config> top_srcdir=<path_to_PostgreSQL_source_tree>
7883
```
84+
###Windows
7985

80-
Once you have`pg_probackup` installed, complete[the setup](https://postgrespro.com/docs/postgrespro/current/app-pgprobackup.html#pg-probackup-install-and-setup).
86+
Currently pg_probackup can be build using only MSVC 2013.
87+
Build PostgreSQL using[pgwininstall](https://github.com/postgrespro/pgwininstall) or[PostgreSQL instruction](https://www.postgresql.org/docs/10/install-windows-full.html) with MSVC 2013.
88+
If zlib support is needed, src/tools/msvc/config.pl must contain path to directory with compiled zlib.[Example](https://gist.githubusercontent.com/gsmol/80989f976ce9584824ae3b1bfb00bd87/raw/240032950d4ac4801a79625dd00c8f5d4ed1180c/gistfile1.txt)
89+
90+
```shell
91+
CALL"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall" amd64
92+
SET PATH=%PATH%;C:\Perl64\bin
93+
SET PATH=%PATH%;C:\msys64\usr\bin
94+
gen_probackup_project.pl C:\path_to_postgresql_source_tree
95+
```
8196

8297
##Documentation
8398

‎src/archive.c‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ do_archive_push(char *wal_file_path, char *wal_file_name, bool overwrite)
2727
charbackup_wal_file_path[MAXPGPATH];
2828
charabsolute_wal_file_path[MAXPGPATH];
2929
charcurrent_dir[MAXPGPATH];
30-
int64system_id;
30+
uint64system_id;
3131
boolis_compress= false;
3232

3333
if (wal_file_name==NULL&&wal_file_path==NULL)
@@ -50,7 +50,7 @@ do_archive_push(char *wal_file_path, char *wal_file_name, bool overwrite)
5050

5151
if(system_id!=instance_config.system_identifier)
5252
elog(ERROR,"Refuse to push WAL segment %s into archive. Instance parameters mismatch."
53-
"Instance '%s' should have SYSTEM_ID =%ldinstead of%ld",
53+
"Instance '%s' should have SYSTEM_ID ="UINT64_FORMAT"instead of"UINT64_FORMAT,
5454
wal_file_name,instance_name,instance_config.system_identifier,
5555
system_id);
5656

‎src/backup.c‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1682,6 +1682,11 @@ wait_wal_lsn(XLogRecPtr lsn, bool is_start_lsn, bool wait_prev_segment)
16821682
(uint32) (lsn >>32), (uint32)lsn,wal_segment_path);
16831683
}
16841684

1685+
if (!stream_wal&&is_start_lsn&&try_count==30)
1686+
elog(WARNING,"By default pg_probackup assume WAL delivery method to be ARCHIVE. "
1687+
"If continius archiving is not set up, use '--stream' option to make autonomous backup. "
1688+
"Otherwise check that continius archiving works correctly.");
1689+
16851690
if (timeout>0&&try_count>timeout)
16861691
{
16871692
if (file_exists)
@@ -2226,9 +2231,9 @@ backup_files(void *arg)
22262231
structstatbuf;
22272232
pgFile*file= (pgFile*)parray_get(arguments->files_list,i);
22282233

2229-
elog(VERBOSE,"Copying file: \"%s\" ",file->path);
22302234
if (!pg_atomic_test_set_flag(&file->lock))
22312235
continue;
2236+
elog(VERBOSE,"Copying file: \"%s\" ",file->path);
22322237

22332238
/* check for interrupt */
22342239
if (interrupted)

‎src/catalog.c‎

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -816,20 +816,6 @@ pgBackupInit(pgBackup *backup)
816816
backup->server_version[0]='\0';
817817
}
818818

819-
/*
820-
* Copy backup metadata from **src** into **dst**.
821-
*/
822-
void
823-
pgBackupCopy(pgBackup*dst,pgBackup*src)
824-
{
825-
pfree(dst->primary_conninfo);
826-
827-
memcpy(dst,src,sizeof(pgBackup));
828-
829-
if (src->primary_conninfo)
830-
dst->primary_conninfo=pstrdup(src->primary_conninfo);
831-
}
832-
833819
/* free pgBackup object */
834820
void
835821
pgBackupFree(void*backup)

‎src/data.c‎

Lines changed: 24 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,8 @@ do_decompress(void* dst, size_t dst_size, void const* src, size_t src_size,
9898
{
9999
caseNONE_COMPRESS:
100100
caseNOT_DEFINED_COMPRESS:
101+
if (errormsg)
102+
*errormsg="Invalid compression algorithm";
101103
return-1;
102104
#ifdefHAVE_LIBZ
103105
caseZLIB_COMPRESS:
@@ -230,8 +232,8 @@ read_page_from_file(pgFile *file, BlockNumber blknum,
230232
return0;
231233
}
232234
else
233-
elog(WARNING,"File: %s, block %u, expected block size %d,"
234-
"but read %lu, try again",
235+
elog(WARNING,"File: %s, block %u, expected block size %u,"
236+
"but read %zu, try again",
235237
file->path,blknum,BLCKSZ,read_len);
236238
}
237239

@@ -380,7 +382,7 @@ prepare_page(backup_files_arg *arguments,
380382
elseif (page_size!=BLCKSZ)
381383
{
382384
free(ptrack_page);
383-
elog(ERROR,"File: %s, block %u, expected block size %d, but read %lu",
385+
elog(ERROR,"File: %s, block %u, expected block size %d, but read %zu",
384386
file->path,absolute_blknum,BLCKSZ,page_size);
385387
}
386388
else
@@ -572,7 +574,7 @@ backup_data_file(backup_files_arg* arguments,
572574
if (file->size %BLCKSZ!=0)
573575
{
574576
fclose(in);
575-
elog(ERROR,"File: %s, invalid file size %lu",file->path,file->size);
577+
elog(ERROR,"File: %s, invalid file size %zu",file->path,file->size);
576578
}
577579

578580
/*
@@ -784,7 +786,7 @@ restore_data_file(const char *to_path, pgFile *file, bool allow_truncate,
784786
read_len=fread(compressed_page.data,1,
785787
MAXALIGN(header.compressed_size),in);
786788
if (read_len!=MAXALIGN(header.compressed_size))
787-
elog(ERROR,"cannot read block %u of \"%s\" read %lu of %d",
789+
elog(ERROR,"cannot read block %u of \"%s\" read %zu of %d",
788790
blknum,file->path,read_len,header.compressed_size);
789791

790792
/*
@@ -824,6 +826,8 @@ restore_data_file(const char *to_path, pgFile *file, bool allow_truncate,
824826

825827
if (write_header)
826828
{
829+
/* We uncompressed the page, so its size is BLCKSZ */
830+
header.compressed_size=BLCKSZ;
827831
if (fio_fwrite(out,&header,sizeof(header))!=sizeof(header))
828832
elog(ERROR,"cannot write header of block %u of \"%s\": %s",
829833
blknum,file->path,strerror(errno));
@@ -1580,19 +1584,23 @@ check_file_pages(pgFile *file, XLogRecPtr stop_lsn, uint32 checksum_version,
15801584
if (read_len==0&&feof(in))
15811585
break;/* EOF found */
15821586
elseif (read_len!=0&&feof(in))
1583-
elog(ERROR,
1587+
elog(WARNING,
15841588
"odd size page found at block %u of \"%s\"",
15851589
blknum,file->path);
15861590
else
1587-
elog(ERROR,"cannot read header of block %u of \"%s\": %s",
1591+
elog(WARNING,"cannot read header of block %u of \"%s\": %s",
15881592
blknum,file->path,strerror(errno_tmp));
1593+
return false;
15891594
}
15901595

15911596
COMP_FILE_CRC32(use_crc32c,crc,&header,read_len);
15921597

15931598
if (header.block<blknum)
1594-
elog(ERROR,"backup is broken at file->path %s block %u",
1599+
{
1600+
elog(WARNING,"backup is broken at file->path %s block %u",
15951601
file->path,blknum);
1602+
return false;
1603+
}
15961604

15971605
blknum=header.block;
15981606

@@ -1608,8 +1616,11 @@ check_file_pages(pgFile *file, XLogRecPtr stop_lsn, uint32 checksum_version,
16081616
read_len=fread(compressed_page.data,1,
16091617
MAXALIGN(header.compressed_size),in);
16101618
if (read_len!=MAXALIGN(header.compressed_size))
1611-
elog(ERROR,"cannot read block %u of \"%s\" read %lu of %d",
1619+
{
1620+
elog(WARNING,"cannot read block %u of \"%s\" read %zu of %d",
16121621
blknum,file->path,read_len,header.compressed_size);
1622+
return false;
1623+
}
16131624

16141625
COMP_FILE_CRC32(use_crc32c,crc,compressed_page.data,read_len);
16151626

@@ -1636,11 +1647,13 @@ check_file_pages(pgFile *file, XLogRecPtr stop_lsn, uint32 checksum_version,
16361647
is_valid= false;
16371648
continue;
16381649
}
1639-
elog(ERROR,"page of file \"%s\" uncompressed to %d bytes. != BLCKSZ",
1650+
elog(WARNING,"page of file \"%s\" uncompressed to %d bytes. != BLCKSZ",
16401651
file->path,uncompressed_size);
1652+
return false;
16411653
}
1654+
16421655
if (validate_one_page(page.data,file,blknum,
1643-
stop_lsn,checksum_version)==PAGE_IS_FOUND_AND_NOT_VALID)
1656+
stop_lsn,checksum_version)==PAGE_IS_FOUND_AND_NOT_VALID)
16441657
is_valid= false;
16451658
}
16461659
else

‎src/delete.c‎

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,7 @@ delete_backup_files(pgBackup *backup)
256256
charpath[MAXPGPATH];
257257
chartimestamp[100];
258258
parray*files;
259+
size_tnum_files;
259260

260261
/*
261262
* If the backup was deleted already, there is nothing to do.
@@ -286,23 +287,16 @@ delete_backup_files(pgBackup *backup)
286287

287288
/* delete leaf node first */
288289
parray_qsort(files,pgFileComparePathDesc);
289-
for (i=0;i<parray_num(files);i++)
290+
num_files=parray_num(files);
291+
for (i=0;i<num_files;i++)
290292
{
291293
pgFile*file= (pgFile*)parray_get(files,i);
292294

293-
/* printprogress */
294-
elog(VERBOSE,"Delete file(%zd/%lu)\"%s\"",i+1,
295-
(unsigned long)parray_num(files),file->path);
295+
if (progress)
296+
elog(INFO,"Progress:(%zd/%zd). Process file\"%s\"",
297+
i+1,num_files,file->path);
296298

297-
if (remove(file->path))
298-
{
299-
if (errno==ENOENT)
300-
elog(VERBOSE,"File \"%s\" is absent",file->path);
301-
else
302-
elog(ERROR,"Cannot remove \"%s\": %s",file->path,
303-
strerror(errno));
304-
return;
305-
}
299+
pgFileDelete(file);
306300
}
307301

308302
parray_walk(files,pgFileFree);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp