Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit651dc0f

Browse files
author
Alexander Korotkov
committed
Add description and howto about incremental backup.
1 parent4890370 commit651dc0f

File tree

1 file changed

+113
-3
lines changed

1 file changed

+113
-3
lines changed

‎README.md

Lines changed: 113 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
1-
pg_arman
2-
========
1+
pg_arman fork from Postgres Professional
2+
========================================
3+
4+
This repository contains fork of pg_arman by Postgres Professional with
5+
block level incremental backup support.
36

47
pg_arman is a backup and recovery manager for PostgreSQL servers able to do
58
differential and full backup as well as restore a cluster to a
@@ -14,7 +17,8 @@ Download
1417
--------
1518

1619
The latest version of this software can be found on the project website at
17-
https://github.com/michaelpq/pg_arman.
20+
https://github.com/postgrespro/pg_arman. Original fork of pg_arman can be
21+
found athttps://github.com/michaelpq/pg_arman.
1822

1923
Installation
2024
------------
@@ -48,6 +52,112 @@ launched in a way similar to common PostgreSQL extensions and modules:
4852

4953
make installcheck
5054

55+
Block level incremental backup
56+
------------------------------
57+
58+
Idea of block level incremental backup is that you may backup only blocks
59+
changed since last full backup. It gives two major benefits: taking backups
60+
faster and making backups smaller.
61+
62+
The major question here is how to get the list of changed blocks. Since
63+
each block contains LSN number, changed blocks could be retrieved by full scan
64+
of all the blocks. But this approach consumes as much server IO as full
65+
backup.
66+
67+
This is why we implemented alternative approaches to retrieve
68+
list of changed blocks.
69+
70+
1. Scan WAL archive and extract changed blocks from it. However, shortcoming
71+
of these approach is requirement to have WAL archive.
72+
73+
2. Track bitmap of changes blocks inside PostgreSQL (ptrack). It introduces
74+
some overhead to PostgreSQL performance. On our experiments it appears to be
75+
less than 3%.
76+
77+
These two approaches were implemented in this fork of pg_arman. The second
78+
approach requires[patch for PostgreSQL](https://gist.github.com/stalkerg/7ce2394c4f3b36f995895190a633b4fa).
79+
80+
Testing block level incremental backup
81+
--------------------------------------
82+
83+
You need build and install[PGPRO9_5_ptrack branch of PostgreSQL](https://github.com/postgrespro/postgrespro/tree/PGPRO9_5_ptrack) or[apply this patch to PostgreSQL 9.5](https://gist.github.com/stalkerg/7ce2394c4f3b36f995895190a633b4fa).
84+
85+
###Retrieving changed blocks from WAL archive
86+
87+
You need to enable WAL archive by adding following lines to postgresql.conf:
88+
89+
```
90+
wal_level = archive
91+
archive_command = 'test ! -f /home/postgres/backup/arman/wal/%f && cp %p /home/postgres/backup/arman/wal/%f'
92+
wal_log_hints = on
93+
```
94+
95+
Example backup (assuming PostgreSQL is running):
96+
```bash
97+
# Init pg_aramn backup folder
98+
pg_arman init -B /home/postgres/backup/pgarman
99+
cat<<__EOF__ >> /home/postgres/backup/pgarman/pg_arman.ini
100+
ARCLOG_PATH = '/home/postgres/backup/arman/wal'
101+
__EOF__
102+
# Make full backup with 2 thread and verbose mode.
103+
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b full -v -j 2
104+
# Validate backup
105+
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
106+
# Show backups information
107+
pg_arman show -B /home/postgres/backup/pgarman
108+
109+
# Now you can insert or update some data in your database
110+
111+
# Then start the incremental backup.
112+
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b page -v -j 2
113+
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
114+
# You should see that increment is really small
115+
pg_arman show -B /home/postgres/backup/pgarman
116+
```
117+
118+
For restore after remove your pgdata you can use:
119+
```
120+
pg_arman restore -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -j 4 --verbose
121+
```
122+
123+
###Retrieving changed blocks from ptrack
124+
125+
The advantage of this approach is that you don't have to save WAL archive. You will need to enable ptrack in postgresql.conf (restart required).
126+
127+
```
128+
ptrack_enable = on
129+
```
130+
131+
Also, some WALs still need to be fetched in order to get consistent backup. pg_arman can fetch them trough the streaming replication protocol. Thus, you also need to[enable streaming replication connection](https://wiki.postgresql.org/wiki/Streaming_Replication).
132+
133+
Example backup (assuming PostgreSQL is running):
134+
```bash
135+
# Init pg_aramn backup folder
136+
pg_arman init -B /home/postgres/backup/pgarman
137+
cat<<__EOF__ >> /home/postgres/backup/pgarman/pg_arman.ini
138+
ARCLOG_PATH = '/home/postgres/backup/arman/wal'
139+
__EOF__
140+
# Make full backup with 2 thread and verbose mode.
141+
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b full -v -j 2 --stream
142+
# Validate backup
143+
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
144+
# Show backups information
145+
pg_arman show -B /home/postgres/backup/pgarman
146+
147+
# Now you can insert or update some data in your database
148+
149+
# Then start the incremental backup.
150+
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b ptrack -v -j 2 --stream
151+
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
152+
# You should see that increment is really small
153+
pg_arman show -B /home/postgres/backup/pgarman
154+
```
155+
156+
For restore after remove your pgdata you can use:
157+
```
158+
pg_arman restore -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -j 4 --verbose --stream
159+
```
160+
51161
License
52162
-------
53163

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp