| F.41. pgpro_bindump — a replication protocol module for backup and restore | ||||
|---|---|---|---|---|
| Prev | Up | Appendix F. Additional Supplied Modules and Extensions Shipped inpostgrespro-std-17-contrib | Home | Next |
F.41. pgpro_bindump — a replication protocol module for backup and restore#
pgpro_bindump is a module designed to implement additional commands for thePostgres Pro replication protocol, specifically for use with thepg_probackup3 utility.pgpro_bindump offers the following benefits:
Simplified communication:pgpro_bindump does not require the SSH protocol and associated configurations to connect to a database server.
Integrated WAL transfer: WAL files are transferred together with data files, reducing the risk of data loss during backups.
Optimized storage: Files are not copied as they are, but are archived using thepg_probackup3 own format,
CBOR(RFC 8949 Concise Binary Object Representation), which enhances the storage efficiency.
F.41.1. Architecture#
A dedicated backend process is created to handle connections from external applications. You can use thepg_probackup3 utility or a custom application to send requests to this backend.
Multiple connections can be established for the same application if needed.
Since thepgpro_bindump module has direct access to the database files, it does not require any additional permissions.
pgpro_bindump uses thelibpgprobackup, along with its built-in encoder module, to send data to the application.
Thewalsender_plugin_libraries plugin has been added to simplify the process of adding and utilizing new replication commands, including custom ones.
F.41.2. Limitations and Requirements#
pgpro_bindump currently has the following limitations and requirements:
libpgprobackup.soandlibpb3_encoder.soare required forpgpro_bindump operation.
F.41.3. Installation and Setup#
Once you completePostgres Pro installation, follow the steps below.
Specify the following parameters in the
postgresql.conffile:shared_preload_libraries = 'pgpro_bindump'wal_level = 'replica' # or 'logical'walsender_plugin_libraries = 'pgpro_bindump'
Restart thePostgres Pro instance.
F.41.4. Replication Commands#
Thepgpro_bindump module implements the following replication commands:
just-noop— a do-nothing command.start_backup— starts the backup process.copy_files— copies files in theCBORformat.stop_backup— stops the backup process.
The backup process runs in the PRO mode — an advanced data source mode that incorporates all available optimizations and enhancements, including the proprietary replication protocol and specialized replication commands.
To send commands from a user application topgpro_bindump and to receive the output data frompgpro_bindump by the application, uselibpq.
Note
An established connection to a runningPostgres Pro server is required.
F.41.4.1. start_backup#
start_backup LABELlabel, INCR_MODEincr_mode, START_LSNstart_lsn, TRANSFER_MODEtransfer_mode,[TRANSFER_BUFFER_SIZEbuffer_size]
Starts the backup process. Sends the following data to the application:
start_lsn— LSN at the start of the backup.start_tli— the timeline ID at the start of the backup.min_xid— the minimal transaction ID at the start of the backup.min_multixact— the minimalmultixactID at the start of the backup.tablespace_list— a list of tablespaces included in the backup.database_list— a list of databases included in the backup.
LABELlabelThe name of the backup.
INCR_MODEincr_modeThe type of incremental backup. Possible values:
PTRACK,DELTA.START_LSNstart_lsnAn
XLogRecPtrvalue representing LSN at the start of the backup.TRANSFER_MODEtransfer_modeThe method to be used for sending data from a server to an application. Possible values:
raw— unpacked data is sent in random blocks of arbitrary size.packed— packed data is sent in blocks of 128 KB with a common header.
packedis the default value.TRANSFER_BUFFER_SIZEbuffer_sizeA numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).
F.41.4.2. copy_files#
copy_files VERIFY_CHECKSUMS [TRUE | FALSE], COMPRESS_ALGcompress_alg, COMPRESS_LVLcompress_lvl, TRANSFER_MODEtransfer_mode,[INCR_MODEincr_mode], [START_LSNstart_lsn], [TRANSFER_BUFFER_SIZEbuffer_size],[WORKER_INFOnumber_of_workers:worker_id]
Forwards blocks of data files and WAL files packaged in theCBOR format.
VERIFY_CHECKSUMSA boolean value (
trueorfalse) indicating whether to verify checksums.COMPRESS_ALGcompress_algThe compression algorithm to be used. Possible values:
zlib,zstd,lz4,none.COMPRESS_LVLcompress_lvlThe compression level. Possible values: 1, 2, 3, 4, 5.
TRANSFER_MODEtransfer_modeThe method to be used for sending data from a server to an application. Possible values:
raw— unpacked data is sent in random blocks of arbitrary size.packed— packed data is sent in blocks of 128 KB with a common header.
packedis the default value.INCR_MODEincr_modeThe type of incremental backup. Possible values:
PTRACK,DELTA.START_LSNstart_lsnAn
XLogRecPtrvalue representing LSN at the start of the backup.TRANSFER_BUFFER_SIZEbuffer_sizeA numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).
WORKER_INFOnumber_of_workers:worker_idThe number of workers and the worker ID.
F.41.4.3. stop_backup#
stop_backup STREAM [TRUE | FALSE], COMPRESS_ALGcompress_alg, COMPRESS_LVLcompress_lvl,[EXTERNALSexternals] [TRANSFER_BUFFER_SIZEbuffer_size], [TRANSFER_MODEtransfer_mode]
Completes the copy process and closes the existing connection. Sends the following data to the application:
stop_lsn— LSN at the end of the backup.stop_tli— the timeline ID at the end of the backup.The
backup.controlfile.A set of WAL files (only if WAL streaming is enabled).
STREAMA boolean value (
trueorfalse) indicating whether WAL streaming is enabled.COMPRESS_ALGcompress_algThe compression algorithm to be used. Possible values:
zlib,zstd,lz4,none.COMPRESS_LVLcompress_lvlThe compression level. Possible values: 1, 2, 3, 4, 5.
EXTERNALSexternalsA list of external directories.
TRANSFER_BUFFER_SIZEbuffer_sizeA numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).
TRANSFER_MODEtransfer_modeThe method to be used for sending data from a server to an application. Possible values:
raw— unpacked data is sent in random blocks of arbitrary size.packed— packed data is sent in blocks of 128 KB with a common header.
packedis the default value.
F.41.5. Authors#
Postgres Professional, Moscow, Russia.