Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitbef47ff

Browse files
committed
Introduce 'bbsink' abstraction to modularize base backup code.
The base backup code has accumulated a healthy number of newfeatures over the years, but it's becoming increasingly difficultto maintain and further enhance that code because there's noreal separation of concerns. For example, the code thatunderstands knows the details of how we send data to the clientusing the libpq protocol is scattered throughout basebackup.c,rather than being centralized in one place.To try to improve this situation, introduce a new 'bbsink' objectwhich acts as a recipient for archives generated during the basebackup progress and also for the backup manifest. This commitintroduces three types of bbsink: a 'copytblspc' bbsink forwards thebackup to the client using one COPY OUT operation per tablespace andanother for the manifest, a 'progress' bbsink performs commandprogress reporting, and a 'throttle' bbsink performs rate-limiting.The 'progress' and 'throttle' bbsink types also forward the data to asuccessor bbsink; at present, the last bbsink in the chain willalways be of type 'copytblspc'. There are plans to add more typesof 'bbsink' in future commits.This abstraction is a bit leaky in the case of progress reporting,but this still seems cleaner than what we had before.Patch by me, reviewed and tested by Andres Freund, Sumanta Mukherjee,Dilip Kumar, Suraj Kharage, Dipesh Pandit, Tushar Ahuja, Mark Dilger,and Jeevan Ladhe.Discussion:https://postgr.es/m/CA+TgmoZGwR=ZVWFeecncubEyPdwghnvfkkdBe9BLccLSiqdf9Q@mail.gmail.comDiscussion:https://postgr.es/m/CA+TgmoZvqk7UuzxsX1xjJRmMGkqoUGYTZLDCH8SmU1xTPr1Xig@mail.gmail.com
1 parentbd807be commitbef47ff

File tree

9 files changed

+1414
-516
lines changed

9 files changed

+1414
-516
lines changed

‎src/backend/replication/Makefile

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,10 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
1717
OBJS =\
1818
backup_manifest.o\
1919
basebackup.o\
20+
basebackup_copy.o\
21+
basebackup_progress.o\
22+
basebackup_sink.o\
23+
basebackup_throttle.o\
2024
repl_gram.o\
2125
slot.o\
2226
slotfuncs.o\

‎src/backend/replication/backup_manifest.c

Lines changed: 9 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
#include"libpq/pqformat.h"
1818
#include"mb/pg_wchar.h"
1919
#include"replication/backup_manifest.h"
20+
#include"replication/basebackup_sink.h"
2021
#include"utils/builtins.h"
2122
#include"utils/json.h"
2223

@@ -310,9 +311,8 @@ AddWALInfoToBackupManifest(backup_manifest_info *manifest, XLogRecPtr startptr,
310311
* Finalize the backup manifest, and send it to the client.
311312
*/
312313
void
313-
SendBackupManifest(backup_manifest_info*manifest)
314+
SendBackupManifest(backup_manifest_info*manifest,bbsink*sink)
314315
{
315-
StringInfoDataprotobuf;
316316
uint8checksumbuf[PG_SHA256_DIGEST_LENGTH];
317317
charchecksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
318318
size_tmanifest_bytes_done=0;
@@ -352,38 +352,28 @@ SendBackupManifest(backup_manifest_info *manifest)
352352
(errcode_for_file_access(),
353353
errmsg("could not rewind temporary file")));
354354

355-
/* Send CopyOutResponse message */
356-
pq_beginmessage(&protobuf,'H');
357-
pq_sendbyte(&protobuf,0);/* overall format */
358-
pq_sendint16(&protobuf,0);/* natts */
359-
pq_endmessage(&protobuf);
360355

361356
/*
362-
* Send CopyData messages.
363-
*
364-
* We choose to read back the data from the temporary file in chunks of
365-
* size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
366-
* size, so it seems to make sense to match that value here.
357+
* Send the backup manifest.
367358
*/
359+
bbsink_begin_manifest(sink);
368360
while (manifest_bytes_done<manifest->manifest_size)
369361
{
370-
charmanifestbuf[BLCKSZ];
371362
size_tbytes_to_read;
372363
size_trc;
373364

374-
bytes_to_read=Min(sizeof(manifestbuf),
365+
bytes_to_read=Min(sink->bbs_buffer_length,
375366
manifest->manifest_size-manifest_bytes_done);
376-
rc=BufFileRead(manifest->buffile,manifestbuf,bytes_to_read);
367+
rc=BufFileRead(manifest->buffile,sink->bbs_buffer,
368+
bytes_to_read);
377369
if (rc!=bytes_to_read)
378370
ereport(ERROR,
379371
(errcode_for_file_access(),
380372
errmsg("could not read from temporary file: %m")));
381-
pq_putmessage('d',manifestbuf,bytes_to_read);
373+
bbsink_manifest_contents(sink,bytes_to_read);
382374
manifest_bytes_done+=bytes_to_read;
383375
}
384-
385-
/* No more data, so send CopyDone message */
386-
pq_putemptymessage('c');
376+
bbsink_end_manifest(sink);
387377

388378
/* Release resources */
389379
BufFileClose(manifest->buffile);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp