The only one configurable option is`ptrack.map_size` (in MB). Default is`-1`, which means`ptrack` is turned off. To completely avoid false positives it is recommended to set`ptrack.map_size` to`1 / 1000` of expected`PGDATA` size (i.e.`1000` for a 1 TB database), since a single 8 byte`ptrack` map record tracks changes in a standard 8 KB PostgreSQL page. To disable`ptrack` and clean up all remaining service files set`ptrack.map_size` to`0`.

The only one configurable option is`ptrack.map_size` (in MB). Default is`-1`, which means`ptrack` is turned off. To completely avoid false positives it is recommended to set`ptrack.map_size` to`1 / 1000` of expected`PGDATA` size (i.e.`1000` for a 1 TB database), since a single 8 byte`ptrack` map record tracks changes in a standard 8 KB PostgreSQL page.

To disable`ptrack` and clean up all remaining service files set`ptrack.map_size` to`0`.

2. The only one production-ready backup utility, that fully supports`ptrack` is[pg_probackup](https://github.com/postgrespro/pg_probackup).

3. Currently, you cannot resize`ptrack` map in runtime, only on postmasterrestart. Also, you will loose all tracked changes, so it is recommended to do so in the maintainance window and accompany this operation with full backup. See[TODO](#TODO) for details.

3. Currently, you cannot resize`ptrack` map in runtime, only on postmasterstart. Also, you will loose all tracked changes, so it is recommended to do so in the maintainance window and accompany this operation with full backup. See[TODO](#TODO) for details.

4. You will need up to`ptrack.map_size * 3` of additional disk space, since`ptrack` uses two additional temporary files for durability purpose. See[TODO](#Architecture) for details.

Briefly, an overhead of using`ptrack` on TPS usually does not exceed a couple of percent (~1-3%) for a database of dozens to hundreds of gigabytes in size, while the backup time scales down linearly with backup size with a coefficient~1. It means that an incremental`ptrack` backup of a database with only 20% of changed pages will be 5 times faster than a full backup. More details[here](benchmarks).

We use a single shared hash table in`ptrack`, which is mapped in memory from the file on disk using`mmap`. Due to the fixed size of the map there may be false positives (when some block is marked as changed without being actually modified), but not false negative results. However, these false postives may be completely eliminated by setting a high enough`ptrack.map_size`.

First target was to measure`ptrack` overhead on TPS due to marking modified pages in the map in memory. We used PostgreSQL 12 cluster of approximately 1 GB size, initialized with`pgbench` on a`tmpfs` partition:

pgbench -i -s 133

Default`pgbench` transaction script[were modified](pgb.sql) to exclude`pgbench_tellers` and`pgbench_branches` updates in order to lower lock contention and make`ptrack` overhead more visible. So`pgbench` was invoked as following:

pgbench -s133 -c40 -j1 -n -P15 -T300 -f pgb.sql

Results:

| ptrack.map_size, MB| 0 (turned off)| 32| 64| 256| 512| 1024|

|---------------------|----------------|----|----|-----|-----|------|

| TPS| 16900| 16890| 16855| 16468| 16490| 16220|

TPS fluctuates in a several percent range around 16500 on the used machine, but in average`ptrack` overhead does not exceed 1-3% for any reasonable`ptrack.map_size`. It only becomes noticeable closer to 1 GB`ptrack.map_size` (~3-4%), which is enough to track changes in the database of up to 1 TB size without false positives.

To test incremental backups speed a fresh cluster were initialized with following DDL:

CREATETABLElarge_test (num1bigint, num2double precision, num3double precision);

CREATETABLElarge_test2 (num1bigint, num2double precision, num3double precision);

These relations were populated with approximately 2 GB of data that way:

INSERT INTO large_test (num1, num2, num3)

SELECT s, random(), random()*142

FROM generate_series(1,20000000) s;

Then a part of one relation was touched with a following query:

UPDATE large_test2SET num3= num3+1WHERE num1<20000000/5;

After that, incremental`ptrack` backups were taken with`pg_probackup` followed by full backups. Tests show that`ptrack_backup_time / full_backup_time ~= ptrack_backup_size / full_backup_size`, i.e. if the only 20% of data were modified, then`ptrack` backup will be 5 times faster than full backup. Thus, the overhead of building`ptrack` map during backup is minimal. Example:

\set aid random(1,100000* :scale)

\set bid random(1,1* :scale)

\set tid random(1,10* :scale)

\set delta random(-5000,5000)

BEGIN;

UPDATE pgbench_accountsSET abalance= abalance+ :deltaWHERE aid= :aid;

SELECT abalanceFROM pgbench_accountsWHERE aid= :aid;

INSERT INTO pgbench_history (tid, bid, aid, delta, mtime)VALUES (:tid, :bid, :aid, :delta,CURRENT_TIMESTAMP);

END;