Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit4783e61

Browse files
committed
Merge branch 'master' into issue_103
2 parents759aefc +535e417 commit4783e61

File tree

5 files changed

+220
-21
lines changed

5 files changed

+220
-21
lines changed

‎Documentation.md

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -453,12 +453,16 @@ Redardless of data checksums been enabled or not, pg_probackup always check page
453453

454454
####External directories
455455

456-
To back up a directory located outside of the data directory, use the optional`--external-dirs` parameter that specifies the path to this directory. If you would like to add more than one external directory, provide several paths separated by colons.
456+
To back up a directory located outside of the data directory, use the optional`--external-dirs` parameter that specifies the path to this directory. If you would like to add more than one external directory, provide several paths separated by colons, on Windows system paths must be separated by semicolon instead.
457457

458-
For example, to include '/etc/dir1/' and '/etc/dir2/' directories into the full backup of your*instance_name* instance that will be stored under the*backup_dir* directory, run:
458+
For example, to include`'/etc/dir1/'` and`'/etc/dir2/'` directories into the full backup of your*instance_name* instance that will be stored under the*backup_dir* directory, run:
459459

460460
pg_probackup backup -B backup_dir --instance instance_name -b FULL --external-dirs=/etc/dir1:/etc/dir2
461461

462+
For example, to include`'C:\dir1\'` and`'C:\dir2\'` directories into the full backup of your*instance_name* instance that will be stored under the*backup_dir* directory on Windows system, run:
463+
464+
pg_probackup backup -B backup_dir --instance instance_name -b FULL --external-dirs=C:\dir1;C:\dir2
465+
462466
pg_probackup creates a separate subdirectory in the backup directory for each external directory. Since external directories included into different backups do not have to be the same, when you are restoring the cluster from an incremental backup, only those directories that belong to this particular backup will be restored. Any external directories stored in the previous backups will be ignored.
463467

464468
To include the same directories into each backup of your instance, you can specify them in the pg_probackup.conf configuration file using the[set-config](#set-config) command with the`--external-dirs` option.
@@ -582,6 +586,14 @@ The typical workflow is as follows:
582586

583587
- Run[backup](#backup) or[restore](#restore) commands with[remote options](#remote-mode-options)**on backup host**. pg_probackup connects to the remote system via SSH and creates a backup locally or restores the previously taken backup on the remote system, respectively.
584588

589+
For example, to create archive full backup using remote mode through SSH connection to user`postgres` on host with address`192.168.0.2` via port`2302`, run:
590+
591+
pg_probackup backup -B backup_dir --instance instance_name -b FULL --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302
592+
593+
For example, to restore latest backup on remote system using remote mode through SSH connection to user`postgres` on host with address`192.168.0.2` via port`2302`, run:
594+
595+
pg_probackup restore -B backup_dir --instance instance_name --remote-user=postgres --remote-host=192.168.0.2 --remote-port=2302
596+
585597
>NOTE: The remote backup mode is currently unavailable for Windows systems.
586598
587599
###Running pg_probackup on Parallel Threads
@@ -953,8 +965,10 @@ Specifies the backup mode to use. Possible values are:
953965
- PAGE — creates an incremental PAGE backup based on the WAL files that have changed since the previous full or incremental backup was taken.
954966
- PTRACK — creates an incremental PTRACK backup tracking page changes on the fly.
955967

956-
-C
957-
--smooth-checkpoint
968+
```
969+
-C
970+
--smooth-checkpoint
971+
```
958972
Spreads out the checkpoint over a period of time. By default, pg_probackup tries to complete the checkpoint as soon as possible.
959973

960974
--stream

‎README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -58,15 +58,15 @@ Regardless of the chosen backup type, all backups taken with `pg_probackup` supp
5858
###Linux Installation
5959
```shell
6060
#DEB Ubuntu|Debian Packages
61-
echo"deb [arch=amd64] http://repo.postgrespro.ru/pg_probackup/deb/$(lsb_release -cs) main-$(lsb_release -cs)"> /etc/apt/sources.list.d/pg_probackup.list
62-
wget -O - http://repo.postgrespro.ru/pg_probackup/keys/GPG-KEY-PG_PROBACKUP| apt-key add -&& apt-get update
63-
apt-get install pg-probackup-{11,10,9.6,9.5}
64-
apt-get install pg-probackup-{11,10,9.6,9.5}-dbg
61+
sudoecho"deb [arch=amd64] http://repo.postgrespro.ru/pg_probackup/deb/$(lsb_release -cs) main-$(lsb_release -cs)"> /etc/apt/sources.list.d/pg_probackup.list
62+
sudowget -O - http://repo.postgrespro.ru/pg_probackup/keys/GPG-KEY-PG_PROBACKUP|sudoapt-key add -&& sudo apt-get update
63+
sudoapt-get install pg-probackup-{11,10,9.6,9.5}
64+
sudoapt-get install pg-probackup-{11,10,9.6,9.5}-dbg
6565

6666
#DEB-SRC Packages
67-
echo"deb-src [arch=amd64] http://repo.postgrespro.ru/pg_probackup/deb/$(lsb_release -cs) main-$(lsb_release -cs)">>\
67+
sudoecho"deb-src [arch=amd64] http://repo.postgrespro.ru/pg_probackup/deb/$(lsb_release -cs) main-$(lsb_release -cs)">>\
6868
/etc/apt/sources.list.d/pg_probackup.list
69-
apt-getsource pg-probackup-{11,10,9.6,9.5}
69+
sudoapt-getsource pg-probackup-{11,10,9.6,9.5}
7070

7171
#RPM Centos Packages
7272
rpm -ivh http://repo.postgrespro.ru/pg_probackup/keys/pg_probackup-repo-centos.noarch.rpm
@@ -112,7 +112,7 @@ gen_probackup_project.pl C:\path_to_postgresql_source_tree
112112

113113
##Documentation
114114

115-
Currently the latest documentation can be found at[github](https://postgrespro.github.io/pg_probackup)
115+
Currently the latest documentation can be found at[github](https://postgrespro.github.io/pg_probackup).
116116
Slightly outdated documentation can be found at[Postgres Pro Enterprise documentation](https://postgrespro.com/docs/postgrespro/current/app-pgprobackup).
117117

118118
##License

‎src/backup.c

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2493,7 +2493,6 @@ StreamLog(void *arg)
24932493
stream_arg->startpos-=stream_arg->startpos %instance_config.xlog_seg_size;
24942494

24952495
/* Initialize timeout */
2496-
stream_stop_timeout=0;
24972496
stream_stop_begin=0;
24982497

24992498
#ifPG_VERSION_NUM >=100000

‎src/pg_probackup.c

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -323,11 +323,11 @@ main(int argc, char *argv[])
323323
||strcmp(argv[1], "-V")==0)
324324
{
325325
#ifdefPGPRO_VERSION
326-
fprintf(stderr,"%s %s (Postgres Pro %s %s)\n",
326+
fprintf(stdout,"%s %s (Postgres Pro %s %s)\n",
327327
PROGRAM_NAME,PROGRAM_VERSION,
328328
PGPRO_VERSION,PGPRO_EDITION);
329329
#else
330-
fprintf(stderr,"%s %s (PostgreSQL %s)\n",
330+
fprintf(stdout,"%s %s (PostgreSQL %s)\n",
331331
PROGRAM_NAME,PROGRAM_VERSION,PG_VERSION);
332332
#endif
333333
exit(0);
@@ -638,12 +638,12 @@ main(int argc, char *argv[])
638638
returndo_show(current.backup_id);
639639
caseDELETE_CMD:
640640
if (delete_expired&&backup_id_string)
641-
elog(ERROR,"You cannot specify --delete-expired and--backup-id options together");
641+
elog(ERROR,"You cannot specify --delete-expired and(-i, --backup-id) options together");
642642
if (merge_expired&&backup_id_string)
643-
elog(ERROR,"You cannot specify --merge-expired and--backup-id options together");
643+
elog(ERROR,"You cannot specify --merge-expired and(-i, --backup-id) options together");
644644
if (!delete_expired&& !merge_expired&& !delete_wal&& !backup_id_string)
645645
elog(ERROR,"You must specify at least one of the delete options: "
646-
"--expired |--wal |--merge-expired |--delete-invalid |--backup_id");
646+
"--delete-expired |--delete-wal |--merge-expired |(-i, --backup-id)");
647647
if (!backup_id_string)
648648
returndo_retention();
649649
else

‎tests/retention.py

Lines changed: 190 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
importos
22
importunittest
33
fromdatetimeimportdatetime,timedelta
4-
from .helpers.ptrack_helpersimportProbackupTest
4+
from .helpers.ptrack_helpersimportProbackupTest,ProbackupException
55
fromtimeimportsleep
66

77

@@ -1264,16 +1264,156 @@ def test_window_error_backups(self):
12641264
self.set_archiving(backup_dir,'node',node)
12651265
node.slow_start()
12661266

1267-
node.pgbench_init(scale=3)
1268-
12691267
# Take FULL BACKUPs
12701268
backup_id_a1=self.backup_node(backup_dir,'node',node)
1271-
page_id_a2=self.backup_node(
1269+
gdb=self.backup_node(
1270+
backup_dir,'node',node,backup_type='page',gdb=True)
1271+
1272+
page_id_a3=self.backup_node(
12721273
backup_dir,'node',node,backup_type='page')
12731274

12741275
# Change FULLb backup status to ERROR
12751276
self.change_backup_status(backup_dir,'node',backup_id_b,'ERROR')
12761277

1278+
# Clean after yourself
1279+
self.del_test_dir(module_name,fname)
1280+
1281+
# @unittest.skip("skip")
1282+
deftest_window_error_backups_1(self):
1283+
"""
1284+
DELTA
1285+
PAGE ERROR
1286+
FULL
1287+
-------window
1288+
"""
1289+
fname=self.id().split('.')[3]
1290+
node=self.make_simple_node(
1291+
base_dir=os.path.join(module_name,fname,'node'),
1292+
initdb_params=['--data-checksums'])
1293+
1294+
backup_dir=os.path.join(self.tmp_path,module_name,fname,'backup')
1295+
self.init_pb(backup_dir)
1296+
self.add_instance(backup_dir,'node',node)
1297+
self.set_archiving(backup_dir,'node',node)
1298+
node.slow_start()
1299+
1300+
# Take FULL BACKUP
1301+
full_id=self.backup_node(backup_dir,'node',node)
1302+
1303+
# Take PAGE BACKUP
1304+
gdb=self.backup_node(
1305+
backup_dir,'node',node,backup_type='page',gdb=True)
1306+
1307+
gdb.set_breakpoint('pg_stop_backup')
1308+
gdb.run_until_break()
1309+
gdb.remove_all_breakpoints()
1310+
gdb._execute('signal SIGINT')
1311+
gdb.continue_execution_until_error()
1312+
1313+
page_id=self.show_pb(backup_dir,'node')[1]['id']
1314+
1315+
# Take DELTA backup
1316+
delta_id=self.backup_node(
1317+
backup_dir,'node',node,backup_type='delta',
1318+
options=['--retention-window=2','--delete-expired'])
1319+
1320+
# Take FULL BACKUP
1321+
full2_id=self.backup_node(backup_dir,'node',node)
1322+
1323+
self.assertEqual(len(self.show_pb(backup_dir,'node')),4)
1324+
1325+
# Clean after yourself
1326+
self.del_test_dir(module_name,fname)
1327+
1328+
# @unittest.skip("skip")
1329+
deftest_window_error_backups_2(self):
1330+
"""
1331+
DELTA
1332+
PAGE ERROR
1333+
FULL
1334+
-------window
1335+
"""
1336+
fname=self.id().split('.')[3]
1337+
node=self.make_simple_node(
1338+
base_dir=os.path.join(module_name,fname,'node'),
1339+
initdb_params=['--data-checksums'])
1340+
1341+
backup_dir=os.path.join(self.tmp_path,module_name,fname,'backup')
1342+
self.init_pb(backup_dir)
1343+
self.add_instance(backup_dir,'node',node)
1344+
self.set_archiving(backup_dir,'node',node)
1345+
node.slow_start()
1346+
1347+
# Take FULL BACKUP
1348+
full_id=self.backup_node(backup_dir,'node',node)
1349+
1350+
# Take PAGE BACKUP
1351+
gdb=self.backup_node(
1352+
backup_dir,'node',node,backup_type='page',gdb=True)
1353+
1354+
gdb.set_breakpoint('pg_stop_backup')
1355+
gdb.run_until_break()
1356+
gdb._execute('signal SIGTERM')
1357+
gdb.continue_execution_until_error()
1358+
1359+
page_id=self.show_pb(backup_dir,'node')[1]['id']
1360+
1361+
# Take DELTA backup
1362+
delta_id=self.backup_node(
1363+
backup_dir,'node',node,backup_type='delta',
1364+
options=['--retention-window=2','--delete-expired'])
1365+
1366+
self.assertEqual(len(self.show_pb(backup_dir,'node')),3)
1367+
1368+
# Clean after yourself
1369+
# self.del_test_dir(module_name, fname)
1370+
1371+
deftest_retention_redundancy_overlapping_chains(self):
1372+
""""""
1373+
fname=self.id().split('.')[3]
1374+
node=self.make_simple_node(
1375+
base_dir=os.path.join(module_name,fname,'node'),
1376+
initdb_params=['--data-checksums'])
1377+
1378+
ifself.get_version(node)<90600:
1379+
self.del_test_dir(module_name,fname)
1380+
returnunittest.skip('Skipped because ptrack support is disabled')
1381+
1382+
backup_dir=os.path.join(self.tmp_path,module_name,fname,'backup')
1383+
self.init_pb(backup_dir)
1384+
self.add_instance(backup_dir,'node',node)
1385+
self.set_archiving(backup_dir,'node',node)
1386+
node.slow_start()
1387+
1388+
self.set_config(
1389+
backup_dir,'node',options=['--retention-redundancy=1'])
1390+
1391+
# Make backups to be purged
1392+
self.backup_node(backup_dir,'node',node)
1393+
self.backup_node(backup_dir,'node',node,backup_type="page")
1394+
1395+
# Make backups to be keeped
1396+
gdb=self.backup_node(backup_dir,'node',node,gdb=True)
1397+
gdb.set_breakpoint('backup_files')
1398+
gdb.run_until_break()
1399+
1400+
sleep(1)
1401+
1402+
self.backup_node(backup_dir,'node',node,backup_type="page")
1403+
1404+
gdb.remove_all_breakpoints()
1405+
gdb.continue_execution_until_exit()
1406+
1407+
self.backup_node(backup_dir,'node',node,backup_type="page")
1408+
1409+
# Purge backups
1410+
log=self.delete_expired(
1411+
backup_dir,'node',options=['--expired','--wal'])
1412+
self.assertEqual(len(self.show_pb(backup_dir,'node')),2)
1413+
1414+
# Clean after yourself
1415+
self.del_test_dir(module_name,fname)
1416+
12771417
deftest_retention_redundancy_overlapping_chains(self):
12781418
""""""
12791419
fname=self.id().split('.')[3]
@@ -1319,3 +1459,49 @@ def test_retention_redundancy_overlapping_chains(self):
13191459

13201460
# Clean after yourself
13211461
self.del_test_dir(module_name,fname)
1462+
1463+
deftest_agressive_retention_window_purge(self):
1464+
"""
1465+
https://github.com/postgrespro/pg_probackup/issues/106
1466+
"""
1467+
fname=self.id().split('.')[3]
1468+
node=self.make_simple_node(
1469+
base_dir=os.path.join(module_name,fname,'node'),
1470+
initdb_params=['--data-checksums'])
1471+
1472+
backup_dir=os.path.join(self.tmp_path,module_name,fname,'backup')
1473+
self.init_pb(backup_dir)
1474+
self.add_instance(backup_dir,'node',node)
1475+
node.slow_start()
1476+
1477+
# Make ERROR incremental backup
1478+
try:
1479+
self.backup_node(backup_dir,'node',node,backup_type='page')
1480+
# we should die here because exception is what we expect to happen
1481+
self.assertEqual(
1482+
1,0,
1483+
"Expecting Error because page backup should not be possible "
1484+
"without valid full backup.\n Output: {0}\n CMD: {1}".format(
1485+
repr(self.output),self.cmd))
1486+
exceptProbackupExceptionase:
1487+
self.assertIn(
1488+
"ERROR: Valid backup on current timeline 1 is not found. "
1489+
"Create new FULL backup before an incremental one.",
1490+
e.message,
1491+
"\n Unexpected Error Message: {0}\n CMD: {1}".format(
1492+
repr(e.message),self.cmd))
1493+
1494+
page_id=self.show_pb(backup_dir,'node')[0]['id']
1495+
1496+
sleep(1)
1497+
1498+
# Make FULL backup
1499+
self.backup_node(
1500+
backup_dir,'node',node,
1501+
options=['--delete-expired','--retention-window=1','--stream'])
1502+
1503+
# Check number of backups
1504+
self.assertEqual(len(self.show_pb(backup_dir,'node')),2)
1505+
1506+
# Clean after yourself
1507+
self.del_test_dir(module_name,fname)

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp