PTRACK allows speed up incremental backups for the huge PostgreSQL databases. PTRACK saves changes of physical blocks in the memory. You can[effectively use](https://postgrespro.github.io/pg_probackup/#pbk-setting-up-ptrack-backups)`PTRACK` engine for taking incremental backups by[pg_probackup](https://github.com/postgrespro/pg_probackup).

Current patch are available for[11](https://github.com/postgrespro/ptrack/blob/master/patches/REL_11_STABLE-ptrack-core.diff),[12](https://github.com/postgrespro/ptrack/blob/master/patches/REL_12_STABLE-ptrack-core.diff),[13](https://github.com/postgrespro/ptrack/blob/master/patches/REL_13_STABLE-ptrack-core.diff),[14](https://github.com/postgrespro/ptrack/blob/master/patches/REL_14_STABLE-ptrack-core.diff),[15](https://github.com/postgrespro/ptrack/blob/master/patches/REL_15_STABLE-ptrack-core.diff)

Current patch are available forPostgreSQL[11](https://github.com/postgrespro/ptrack/blob/master/patches/REL_11_STABLE-ptrack-core.diff),[12](https://github.com/postgrespro/ptrack/blob/master/patches/REL_12_STABLE-ptrack-core.diff),[13](https://github.com/postgrespro/ptrack/blob/master/patches/REL_13_STABLE-ptrack-core.diff),[14](https://github.com/postgrespro/ptrack/blob/master/patches/REL_14_STABLE-ptrack-core.diff),[15](https://github.com/postgrespro/ptrack/blob/master/patches/REL_15_STABLE-ptrack-core.diff)

Enterprise PTRACK are part of[Postgres Pro Enterprise](https://postgrespro.ru/products/postgrespro/enterprise) and share posibility to track more than 100 000 tables and indexes per time without speed degradation with[CFS (compressed file system)](https://postgrespro.ru/docs/enterprise/current/cfs).

Benchmarks are x5 time faster and useful for ERP and DWH with huge amounth of tables and relations between them.

Enterprise PTRACK are part of[Postgres Pro Backup Enterprise](https://postgrespro.ru/products/postgrespro/enterprise) and offers the capability to track more than 100 000 tables and indexes concurrently without any degradation in speed[CFS (compressed file system)](https://postgrespro.ru/docs/enterprise/current/cfs).

According to benchmarks, it operated up to 5 times faster and useful for ERP and DWH with huge amount of tables and relations between them.

2. The only one production-ready backup utility, that fully supports`PTRACK` is[pg_probackup](https://github.com/postgrespro/pg_probackup).

3. You cannot resize`PTRACK` map in runtime, only on postmaster start. Also, you willloose all tracked changes, so it is recommended to do so in the maintainance window and accompany this operation with full backup.

3. You cannot resize`PTRACK` map in runtime, only on postmaster start. Also, you willlose all tracked changes, so it is recommended to do so in the maintainance window and accompany this operation with full backup.

4. You will need up to`ptrack.map_size * 2` of additional disk space, since`PTRACK` uses additional temporary file for durability purpose. See[Architecture section](#Architecture) for details.

It is designed toallow false positives (i.e. block/page is marked in the`PTRACK` map, but actually has not been changed), butto neverallow false negatives (i.e. loosingany`PGDATA` changes, excepting hint-bits).

It is designed topermit false positives (i.e., block/page is markedas alteredin the`PTRACK` map when it hasn't actually been changed), butit nevertolerates false negatives (i.e., it never losesany PGDATA modifications, barring hint-bits).

Currently,`PTRACK` codebase issplit between small PostgreSQL core patch and extension.All public SQL API methods and main engine areplaced in the`PTRACK` extension,while the core patchcontainsonlycertainhooks and modifies binary utilities toignore`ptrack.map.*` files.

At present, thePTRACK codebase isdivided betweenasmall PostgreSQL core patch andanextension.The public SQL API methods andthemain engine arehoused in the`PTRACK` extension,whereas the core patch onlyincludes specifichooks and modifies binary utilities todisregardptrack.map.* files.

Weuse a single shared hash table in`PTRACK`. Due to the fixed size of the map theremay be false positives (whensome block is marked as changed withoutbeing actually modified), butnotfalsenegative results. However, these falsepostives may be completelyeliminated by setting a highenough`ptrack.map_size`.

In`PTRACK`, weuse a single shared hash table. Due to the fixed size of the map, therecan be false positives (whena block is marked as changed withoutactual modification), but falsenegatives are not allowed. Nevertheless, these falsepositives can be completelyremoved by setting asufficientlyhigh ptrack.map_size.

All reads/writes aremade using atomic operations on`uint64` entries,so the mapiscompletely lockless duringthe normalPostgreSQL operation.Because we do notuse locks for read/write access,`PTRACK`keeps a map (`ptrack.map`) since the last checkpointintact and usesup to 1additional temporary file:

All reads/writes areperformed using atomic operations on uint64 entries,making the map completely lockless duringstandardPostgreSQL operation.Since we do notutilize locks for read/write access,`PTRACK`maintains a map (ptrack.map) from the last checkpointunaltered and usesa maximum of oneadditional temporary file.

* temporary file`ptrack.map.tmp` to durably replace`ptrack.map` during checkpoint.

Feel free to[send pull requests](https://github.com/postgrespro/ptrack/compare),[fill up issues](https://github.com/postgrespro/ptrack/issues/new).

See also the list of[authors](AUTHORS.md) who participated in this project.

Everything is tested automatically with[travis-ci.com](https://travis-ci.com/postgrespro/ptrack) and[codecov.io](https://codecov.io/gh/postgrespro/ptrack), but you can also run tests locally via`Docker`:

Available test modes (`MODE`) are`basic` (default) and`paranoia` (per-block checksum comparison of`PGDATA` content before and after backup-restore process). Available test cases (`TEST_CASE`) are`tap` (minimalistic PostgreSQL[tap test](https://github.com/postgrespro/ptrack/blob/master/t/001_basic.pl)),`all` or any specific[pg_probackup test](https://github.com/postgrespro/pg_probackup/blob/master/tests/ptrack.py), e.g.`test_ptrack_simple`.

PTRACK development is supported by Postgres Professional

You can ask any question about contributing or usage in[Russian chat](https://t.me/pg_probackup) or[Internation chat](https://t.me/pg_probackup_eng) of[pg_probackup](https://github.com/postgrespro/pg_probackup).