From f0cc225de08fb44a26ae6adf1b5898a1ca118667 Mon Sep 17 00:00:00 2001 From: Mario Gonzalez Date: Mon, 13 Mar 2023 11:03:02 -0300 Subject: [PATCH 1/3] Adding new mode to clone standbys. repmgr v5.4.0 supports this new mode called `pg_backupapi` which is enabled by defining `pg_backupapi_host` in repmgr.conf. --- doc/cloning-standbys.xml | 97 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/doc/cloning-standbys.xml b/doc/cloning-standbys.xml index fc5a18ab..0b214565 100644 --- a/doc/cloning-standbys.xml +++ b/doc/cloning-standbys.xml @@ -227,6 +227,103 @@ description = "Main cluster" + + Using Barman through its API (pg-backup-api) + + cloning + pg-backup-api + + + + This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to integrate further with Barman and let it + handle all writes. This also reduces the ssh keys you need to share because, as long as you have access to + the API service, you could start to perform recoveries right away. You just need to tell Barman what is + the backup you need and in which node to want it to be restored. + + + + In order to enable pg_backupapi mode support for repmgr standby clone, + you need the following lines in repmgr.conf: + + pg_backupapi_host: Where pg-backup-api is hosted + pg_backupapi_node_name: Name of the server as understood by Barman + pg_backupapi_remote_ssh_command: How Barman will be connecting as to the node + pg_backupapi_backup_id: ID of the existing backup you need to restore + + + This is an example of how repmgr.conf would look like: + + + pg_backupapi_host = '192.168.122.154' + pg_backupapi_node_name = 'burrito' + pg_backupapi_remote_ssh_command = 'ssh john_doe@192.168.122.1' + pg_backupapi_backup_id = '20230223T093201' + + + + + pg_backupapi_host is the variable name that enables this mode, and when you set it, + all the rest of the above variables are required. Also, remember that this service is just an interface + between Barman and repmgr, hence if something fails during a recovery, you should check Barman's logs upon + why the process couldn't finish properly. + + + + + Despite in Barman you can define shortcuts ID like "lastest" or "oldest", they are not supported for the + time being in pg-backup-api. This shortcuts will be supported in a future release. + + + + + This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped + the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary + is listening to port 6001 upon localhost: + + +$ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost +NOTICE: destination directory "/home/mario/nodes/node_3/data" provided +INFO: Attempting to use `pg_backupapi` new restore mode +INFO: connecting to source node +DETAIL: connection string is: user=repmgr port=6001 host=localhost +DETAIL: current installation size is 8541 MB +DEBUG: 1 node records returned by source node +DEBUG: connecting to: "user=repmgr dbname=repmgr host=localhost port=6001 connect_timeout=2 fallback_application_name=repmgr options=-csearch_path=" +DEBUG: upstream_node_id determined as 1 +INFO: Attempting to use `pg_backupapi` new restore mode +INFO: replication slot usage not requested; no replication slot will be set up for this standby +NOTICE: starting backup (using pg_backupapi)... +INFO: Success creating the task: operation id '20230309T150647' +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +Incorrect reply received for that operation ID. +INFO: Retrying... +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status DONE +NOTICE: standby clone (from pg_backupapi) complete +NOTICE: you can now start your PostgreSQL server +HINT: for example: pg_ctl -D /home/mario/nodes/node_3/data start +HINT: after starting the server, you need to register this standby with "repmgr standby register" + + + + + + Cloning and replication slots From 26cfb561701c78a638d8a2c3412da8606c89e45b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mart=C3=ADn=20Marqu=C3=A9s?= Date: Tue, 14 Mar 2023 10:14:16 +0100 Subject: [PATCH 2/3] New sect2 block has to be inside the sect1. The closing tag has to go at the end. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Martín Marqués --- doc/cloning-standbys.xml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/cloning-standbys.xml b/doc/cloning-standbys.xml index 0b214565..39321ad8 100644 --- a/doc/cloning-standbys.xml +++ b/doc/cloning-standbys.xml @@ -225,7 +225,6 @@ description = "Main cluster" - Using Barman through its API (pg-backup-api) @@ -322,7 +321,7 @@ HINT: after starting the server, you need to register this standby with "repmgr - + Cloning and replication slots From 43b8a5f65f5ff2edd37d5dbe39a953949879d37b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mart=C3=ADn=20Marqu=C3=A9s?= Date: Tue, 14 Mar 2023 11:39:22 +0100 Subject: [PATCH 3/3] Various improvements and fixes to the pg-backup-api restore section on the docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I added a paragraph at the beginning of the section on where to look for instructions on how to install the pg-backup-api rest API. Fixed typos and some gramatical changes. Also reworded the first paragraph (which is now the second one). Signed-off-by: Martín Marqués --- doc/cloning-standbys.xml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/doc/cloning-standbys.xml b/doc/cloning-standbys.xml index 39321ad8..4f4d4b00 100644 --- a/doc/cloning-standbys.xml +++ b/doc/cloning-standbys.xml @@ -234,10 +234,17 @@ description = "Main cluster" - This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to integrate further with Barman and let it - handle all writes. This also reduces the ssh keys you need to share because, as long as you have access to - the API service, you could start to perform recoveries right away. You just need to tell Barman what is - the backup you need and in which node to want it to be restored. + You can find information on how to install and setup pg-backup-api in + the pg-backup-api + documentation. + + + + This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to further integrate with Barman letting Barman + handle the restore. This also reduces the ssh keys that need to share between the backup and postgres nodes. + As long as you have access to the API service by HTTP calls, you could perform recoveries right away. + You just need to instruct Barman through the API which backup you need and on which node the backup needs to + to be restored on. @@ -269,15 +276,15 @@ description = "Main cluster" - Despite in Barman you can define shortcuts ID like "lastest" or "oldest", they are not supported for the - time being in pg-backup-api. This shortcuts will be supported in a future release. + Despite in Barman you can define shortcuts like "lastest" or "oldest", they are not supported for the + time being in pg-backup-api. These shortcuts will be supported in a future release. This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary - is listening to port 6001 upon localhost: + is listening on localhost's port 6001: $ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost