From 7af74c4fda2be7a17bcabe55e199c2b1f8cb7a75 Mon Sep 17 00:00:00 2001 From: Mario Gonzalez Date: Mon, 13 Mar 2023 11:03:02 -0300 Subject: [PATCH 1/6] Adding new mode to clone standbys. repmgr v5.4.0 supports this new mode called `pg_backupapi` which is enabled by defining `pg_backupapi_host` in repmgr.conf. --- doc/cloning-standbys.xml | 97 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/doc/cloning-standbys.xml b/doc/cloning-standbys.xml index 4f4d4b00..bac614a4 100644 --- a/doc/cloning-standbys.xml +++ b/doc/cloning-standbys.xml @@ -330,6 +330,103 @@ HINT: after starting the server, you need to register this standby with "repmgr + + Using Barman through its API (pg-backup-api) + + cloning + pg-backup-api + + + + This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to integrate further with Barman and let it + handle all writes. This also reduces the ssh keys you need to share because, as long as you have access to + the API service, you could start to perform recoveries right away. You just need to tell Barman what is + the backup you need and in which node to want it to be restored. + + + + In order to enable pg_backupapi mode support for repmgr standby clone, + you need the following lines in repmgr.conf: + + pg_backupapi_host: Where pg-backup-api is hosted + pg_backupapi_node_name: Name of the server as understood by Barman + pg_backupapi_remote_ssh_command: How Barman will be connecting as to the node + pg_backupapi_backup_id: ID of the existing backup you need to restore + + + This is an example of how repmgr.conf would look like: + + + pg_backupapi_host = '192.168.122.154' + pg_backupapi_node_name = 'burrito' + pg_backupapi_remote_ssh_command = 'ssh john_doe@192.168.122.1' + pg_backupapi_backup_id = '20230223T093201' + + + + + pg_backupapi_host is the variable name that enables this mode, and when you set it, + all the rest of the above variables are required. Also, remember that this service is just an interface + between Barman and repmgr, hence if something fails during a recovery, you should check Barman's logs upon + why the process couldn't finish properly. + + + + + Despite in Barman you can define shortcuts ID like "lastest" or "oldest", they are not supported for the + time being in pg-backup-api. This shortcuts will be supported in a future release. + + + + + This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped + the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary + is listening to port 6001 upon localhost: + + +$ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost +NOTICE: destination directory "/home/mario/nodes/node_3/data" provided +INFO: Attempting to use `pg_backupapi` new restore mode +INFO: connecting to source node +DETAIL: connection string is: user=repmgr port=6001 host=localhost +DETAIL: current installation size is 8541 MB +DEBUG: 1 node records returned by source node +DEBUG: connecting to: "user=repmgr dbname=repmgr host=localhost port=6001 connect_timeout=2 fallback_application_name=repmgr options=-csearch_path=" +DEBUG: upstream_node_id determined as 1 +INFO: Attempting to use `pg_backupapi` new restore mode +INFO: replication slot usage not requested; no replication slot will be set up for this standby +NOTICE: starting backup (using pg_backupapi)... +INFO: Success creating the task: operation id '20230309T150647' +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +Incorrect reply received for that operation ID. +INFO: Retrying... +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status IN_PROGRESS +INFO: status DONE +NOTICE: standby clone (from pg_backupapi) complete +NOTICE: you can now start your PostgreSQL server +HINT: for example: pg_ctl -D /home/mario/nodes/node_3/data start +HINT: after starting the server, you need to register this standby with "repmgr standby register" + + + + + + Cloning and replication slots From 7d45ab04b3e778a4921292f8995c013e99340216 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mart=C3=ADn=20Marqu=C3=A9s?= Date: Tue, 14 Mar 2023 10:14:16 +0100 Subject: [PATCH 2/6] New sect1 block has to be inside the sect1. The closing tag has to go at the end. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Martín Marqués --- doc/cloning-standbys.xml | 106 +-------------------------------------- 1 file changed, 1 insertion(+), 105 deletions(-) diff --git a/doc/cloning-standbys.xml b/doc/cloning-standbys.xml index bac614a4..39321ad8 100644 --- a/doc/cloning-standbys.xml +++ b/doc/cloning-standbys.xml @@ -226,110 +226,6 @@ description = "Main cluster" - - Using Barman through its API (pg-backup-api) - - cloning - pg-backup-api - - - - You can find information on how to install and setup pg-backup-api in - the pg-backup-api - documentation. - - - - This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to further integrate with Barman letting Barman - handle the restore. This also reduces the ssh keys that need to share between the backup and postgres nodes. - As long as you have access to the API service by HTTP calls, you could perform recoveries right away. - You just need to instruct Barman through the API which backup you need and on which node the backup needs to - to be restored on. - - - - In order to enable pg_backupapi mode support for repmgr standby clone, - you need the following lines in repmgr.conf: - - pg_backupapi_host: Where pg-backup-api is hosted - pg_backupapi_node_name: Name of the server as understood by Barman - pg_backupapi_remote_ssh_command: How Barman will be connecting as to the node - pg_backupapi_backup_id: ID of the existing backup you need to restore - - - This is an example of how repmgr.conf would look like: - - - pg_backupapi_host = '192.168.122.154' - pg_backupapi_node_name = 'burrito' - pg_backupapi_remote_ssh_command = 'ssh john_doe@192.168.122.1' - pg_backupapi_backup_id = '20230223T093201' - - - - - pg_backupapi_host is the variable name that enables this mode, and when you set it, - all the rest of the above variables are required. Also, remember that this service is just an interface - between Barman and repmgr, hence if something fails during a recovery, you should check Barman's logs upon - why the process couldn't finish properly. - - - - - Despite in Barman you can define shortcuts like "lastest" or "oldest", they are not supported for the - time being in pg-backup-api. These shortcuts will be supported in a future release. - - - - - This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped - the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary - is listening on localhost's port 6001: - - -$ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost -NOTICE: destination directory "/home/mario/nodes/node_3/data" provided -INFO: Attempting to use `pg_backupapi` new restore mode -INFO: connecting to source node -DETAIL: connection string is: user=repmgr port=6001 host=localhost -DETAIL: current installation size is 8541 MB -DEBUG: 1 node records returned by source node -DEBUG: connecting to: "user=repmgr dbname=repmgr host=localhost port=6001 connect_timeout=2 fallback_application_name=repmgr options=-csearch_path=" -DEBUG: upstream_node_id determined as 1 -INFO: Attempting to use `pg_backupapi` new restore mode -INFO: replication slot usage not requested; no replication slot will be set up for this standby -NOTICE: starting backup (using pg_backupapi)... -INFO: Success creating the task: operation id '20230309T150647' -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -Incorrect reply received for that operation ID. -INFO: Retrying... -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status IN_PROGRESS -INFO: status DONE -NOTICE: standby clone (from pg_backupapi) complete -NOTICE: you can now start your PostgreSQL server -HINT: for example: pg_ctl -D /home/mario/nodes/node_3/data start -HINT: after starting the server, you need to register this standby with "repmgr standby register" - - - - - - Using Barman through its API (pg-backup-api) @@ -425,7 +321,7 @@ HINT: after starting the server, you need to register this standby with "repmgr - + Cloning and replication slots From afdeb5d7b2948bb486144b381f1f4c5e653f0c6a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mart=C3=ADn=20Marqu=C3=A9s?= Date: Tue, 14 Mar 2023 11:39:22 +0100 Subject: [PATCH 3/6] Various improvements and fixes to the pg-backup-api restore section on the docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I added a paragraph at the beginning of the section on where to look for instructions on how to install the pg-backup-api rest API. Fixed typos and some gramatical changes. Also reworded the first paragraph (which is now the second one). Signed-off-by: Martín Marqués --- doc/cloning-standbys.xml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/doc/cloning-standbys.xml b/doc/cloning-standbys.xml index 39321ad8..4f4d4b00 100644 --- a/doc/cloning-standbys.xml +++ b/doc/cloning-standbys.xml @@ -234,10 +234,17 @@ description = "Main cluster" - This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to integrate further with Barman and let it - handle all writes. This also reduces the ssh keys you need to share because, as long as you have access to - the API service, you could start to perform recoveries right away. You just need to tell Barman what is - the backup you need and in which node to want it to be restored. + You can find information on how to install and setup pg-backup-api in + the pg-backup-api + documentation. + + + + This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to further integrate with Barman letting Barman + handle the restore. This also reduces the ssh keys that need to share between the backup and postgres nodes. + As long as you have access to the API service by HTTP calls, you could perform recoveries right away. + You just need to instruct Barman through the API which backup you need and on which node the backup needs to + to be restored on. @@ -269,15 +276,15 @@ description = "Main cluster" - Despite in Barman you can define shortcuts ID like "lastest" or "oldest", they are not supported for the - time being in pg-backup-api. This shortcuts will be supported in a future release. + Despite in Barman you can define shortcuts like "lastest" or "oldest", they are not supported for the + time being in pg-backup-api. These shortcuts will be supported in a future release. This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary - is listening to port 6001 upon localhost: + is listening on localhost's port 6001: $ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost From 20f39e2fbea36216b34542f1f878fa67ad4067ae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mart=C3=ADn=20Marqu=C3=A9s?= Date: Fri, 24 Mar 2023 15:05:03 +0000 Subject: [PATCH 4/6] Add release notes for repmgr release 5.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This release added support for pg-backup-api. Here we add the release notes which expose the new feature. Signed-off-by: Martín Marqués --- HISTORY | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/HISTORY b/HISTORY index 4211785c..c2d37051 100644 --- a/HISTORY +++ b/HISTORY @@ -1,4 +1,4 @@ -5.4.1 2023-??-?? +5.4.1 2023-07-04 repmgrd: ensure witness node metadata is updated (Ian) 5.4.0 2023-03-16 From 440ba5fbb40283b14a9e235f2d519c323bfa1f18 Mon Sep 17 00:00:00 2001 From: Mario Gonzalez Date: Mon, 17 Apr 2023 11:13:28 -0400 Subject: [PATCH 5/6] configure.in: bumping to 5.4.1 release. This release contains all changes related to the new pg-backupapi mode along with the related documentation. In the previous release, we didn't include any documentation at all, just the required C code. --- configure.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.in b/configure.in index 00d4b63c..fa3ea379 100644 --- a/configure.in +++ b/configure.in @@ -1,4 +1,4 @@ -AC_INIT([repmgr],[5.4.1],[repmgr@googlegroups.com],[repmgr],[https://repmgr.org/]) +AC_INIT([repmgr],[5.4.1],[repmgr@googlegroups.com],[repmgr],[https://www.repmgr.org/]) AC_COPYRIGHT([Copyright (c) 2010-2021, EnterpriseDB Corporation]) From 890cbd9fcadd1ad9a2e58e2aa729fd05d3b328b7 Mon Sep 17 00:00:00 2001 From: Sidorov Pavel Date: Tue, 4 Apr 2023 11:54:59 +0300 Subject: [PATCH 6/6] Update README.md upd upper version according to https://repmgr.org --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ea5e0285..d97c2e9e 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ replication, and perform administrative tasks such as failover or switchover operations. The most recent `repmgr` version (5.3.2) supports all PostgreSQL versions from -9.5 to 14. PostgreSQL 9.4 is also supported, with some restrictions. +9.5 to 15. PostgreSQL 9.4 is also supported, with some restrictions. `repmgr` is distributed under the GNU GPL 3 and maintained by EnterpriseDB.