Compare commits

..

8 Commits

Author SHA1 Message Date
Martín Marqués
9a68206b4a Various improvements and fixes to the pg-backup-api restore section on the docs
I added a paragraph at the beginning of the section on where to look for
instructions on how to install the pg-backup-api rest API.

Fixed typos and some gramatical changes. Also reworded the first paragraph
(which is now the second one).

Signed-off-by: Martín Marqués <martin.marques@enterprisedb.com>
2023-03-24 18:25:27 +00:00
Martín Marqués
31af938354 New sect2 block has to be inside the sect1. The closing tag has to go at the end.
Signed-off-by: Martín Marqués <martin.marques@enterprisedb.com>
2023-03-24 18:25:12 +00:00
Mario Gonzalez
061025b62f Adding new mode to clone standbys.
repmgr v5.4.0 supports this new mode called `pg_backupapi` which is
enabled by defining `pg_backupapi_host` in repmgr.conf.
2023-03-24 18:24:56 +00:00
Martín Marqués
02d8e0c808 Merge pull request #799 from EnterpriseDB/validate-repmgr-options
Validate repmgr options
2023-03-08 07:13:45 -03:00
Mario Gonzalez
167d166ae8 Validate options when using pg_backupapi mode.
When `pg_backupapi` is enabled, we should validate if the remote ssh
command, node name and backup_id are also defined.
2023-03-07 16:51:25 -03:00
Mario Gonzalez
03c2ae1bd8 Connecting to default pg-backup-api port: 7480/TCP 2023-03-07 16:50:02 -03:00
Martín Marqués
4021037d38 Merge pull request #797 from EnterpriseDB/connect-to-pg-backup-api
Connect to pg backup api
2023-03-07 14:34:54 -03:00
Ian Barwick
7cd7566409 doc: update README
Remove reference to non-existent "scripts/" directory.
2023-03-07 08:36:00 +09:00
5 changed files with 136 additions and 4 deletions

View File

@@ -40,7 +40,6 @@ Directories
- `contrib/`: additional utilities - `contrib/`: additional utilities
- `doc/`: DocBook-based documentation files - `doc/`: DocBook-based documentation files
- `expected/`: expected regression test output - `expected/`: expected regression test output
- `scripts/`: example scripts
- `sql/`: regression test input - `sql/`: regression test input
@@ -70,7 +69,6 @@ news are always welcome.
Thanks from the repmgr core team. Thanks from the repmgr core team.
* Ian Barwick
* Jaime Casanova * Jaime Casanova
* Abhijit Menon-Sen * Abhijit Menon-Sen
* Simon Riggs * Simon Riggs

View File

@@ -225,6 +225,109 @@ description = "Main cluster"
</note> </note>
</sect2> </sect2>
<sect2 id="cloning-from-barman-pg_backupapi-mode" xreflabel="Using Barman through its API (pg-backup-api)">
<title>Using Barman through its API (pg-backup-api)</title>
<indexterm>
<primary>cloning</primary>
<secondary>pg-backup-api</secondary>
</indexterm>
<para>
You can find information on how to install and setup pg-backup-api in
<ulink url="https://www.enterprisedb.com/docs/supported-open-source/barman/pg-backup-api/">the pg-backup-api
documentation</ulink>.
</para>
<para>
This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to further integrate with Barman letting Barman
handle the restore. This also reduces the ssh keys that need to share between the backup and postgres nodes.
As long as you have access to the API service by HTTP calls, you could perform recoveries right away.
You just need to instruct Barman through the API which backup you need and on which node the backup needs to
to be restored on.
</para>
<para>
In order to enable <literal>pg_backupapi mode</literal> support for <command>repmgr standby clone</command>,
you need the following lines in repmgr.conf:
<itemizedlist spacing="compact" mark="bullet">
<listitem><para>pg_backupapi_host: Where pg-backup-api is hosted</para></listitem>
<listitem><para>pg_backupapi_node_name: Name of the server as understood by Barman</para></listitem>
<listitem><para>pg_backupapi_remote_ssh_command: How Barman will be connecting as to the node</para></listitem>
<listitem><para>pg_backupapi_backup_id: ID of the existing backup you need to restore</para></listitem>
</itemizedlist>
This is an example of how repmgr.conf would look like:
<programlisting>
pg_backupapi_host = '192.168.122.154'
pg_backupapi_node_name = 'burrito'
pg_backupapi_remote_ssh_command = 'ssh john_doe@192.168.122.1'
pg_backupapi_backup_id = '20230223T093201'
</programlisting>
</para>
<para>
<literal>pg_backupapi_host</literal> is the variable name that enables this mode, and when you set it,
all the rest of the above variables are required. Also, remember that this service is just an interface
between Barman and repmgr, hence if something fails during a recovery, you should check Barman's logs upon
why the process couldn't finish properly.
</para>
<note>
<simpara>
Despite in Barman you can define shortcuts like "lastest" or "oldest", they are not supported for the
time being in pg-backup-api. These shortcuts will be supported in a future release.
</simpara>
</note>
<para>
This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped
the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary
is listening on localhost's port 6001:
<programlisting>
$ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost
NOTICE: destination directory "/home/mario/nodes/node_3/data" provided
INFO: Attempting to use `pg_backupapi` new restore mode
INFO: connecting to source node
DETAIL: connection string is: user=repmgr port=6001 host=localhost
DETAIL: current installation size is 8541 MB
DEBUG: 1 node records returned by source node
DEBUG: connecting to: "user=repmgr dbname=repmgr host=localhost port=6001 connect_timeout=2 fallback_application_name=repmgr options=-csearch_path="
DEBUG: upstream_node_id determined as 1
INFO: Attempting to use `pg_backupapi` new restore mode
INFO: replication slot usage not requested; no replication slot will be set up for this standby
NOTICE: starting backup (using pg_backupapi)...
INFO: Success creating the task: operation id '20230309T150647'
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
Incorrect reply received for that operation ID.
INFO: Retrying...
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status DONE
NOTICE: standby clone (from pg_backupapi) complete
NOTICE: you can now start your PostgreSQL server
HINT: for example: pg_ctl -D /home/mario/nodes/node_3/data start
HINT: after starting the server, you need to register this standby with "repmgr standby register"
</programlisting>
</para>
</sect2> <!--END cloning-from-barman-pg_backupapi-mode !-->
</sect1> </sect1>
<sect1 id="cloning-replication-slots" xreflabel="Cloning and replication slots"> <sect1 id="cloning-replication-slots" xreflabel="Cloning and replication slots">

View File

@@ -50,7 +50,7 @@ size_t receive_operations_cb(void *content, size_t size, size_t nmemb, char *buf
} }
char * define_base_url(operation_task *task) { char * define_base_url(operation_task *task) {
char *format = "http://%s:80/servers/%s/operations"; char *format = "http://%s:7480/servers/%s/operations";
char *url = malloc(MAX_BUFFER_LENGTH); char *url = malloc(MAX_BUFFER_LENGTH);
snprintf(url, MAX_BUFFER_LENGTH-1, format, task->host, task->node_name); snprintf(url, MAX_BUFFER_LENGTH-1, format, task->host, task->node_name);

View File

@@ -7795,6 +7795,7 @@ run_pg_backupapi(t_node_info *local_node_record)
CURL *curl = curl_easy_init(); CURL *curl = curl_easy_init();
CURLcode ret; CURLcode ret;
check_pg_backupapi_standby_clone_options();
task->host = malloc(strlen(config_file_options.pg_backupapi_host)+1); task->host = malloc(strlen(config_file_options.pg_backupapi_host)+1);
task->remote_ssh_command = malloc(strlen(config_file_options.pg_backupapi_remote_ssh_command)+1); task->remote_ssh_command = malloc(strlen(config_file_options.pg_backupapi_remote_ssh_command)+1);
@@ -7858,6 +7859,36 @@ run_pg_backupapi(t_node_info *local_node_record)
return r; return r;
} }
/*
* pg_backupapi mode is enabled when config_file_options.pg_backupapi_host is set hence, we
* should also check the other required variables too.
*/
void check_pg_backupapi_standby_clone_options() {
bool error = false;
if (*config_file_options.pg_backupapi_remote_ssh_command == '\0') {
log_hint("Check config: remote ssh command is required");
error = true;
}
if (*config_file_options.pg_backupapi_node_name == '\0') {
log_hint("Check config: node name is required");
error = true;
}
if (*config_file_options.pg_backupapi_backup_id == '\0') {
log_hint("Check config: backup_id is required");
error = true;
}
if (error == true) {
log_error("Please fix the errors and try again");
exit(ERR_BAD_CONFIG);
}
}
static char * static char *
make_barman_ssh_command(char *buf) make_barman_ssh_command(char *buf)

View File

@@ -30,6 +30,6 @@ extern void do_standby_help(void);
extern bool do_standby_follow_internal(PGconn *primary_conn, PGconn *follow_target_conn, t_node_info *follow_target_node_record, PQExpBufferData *output, int general_error_code, int *error_code); extern bool do_standby_follow_internal(PGconn *primary_conn, PGconn *follow_target_conn, t_node_info *follow_target_node_record, PQExpBufferData *output, int general_error_code, int *error_code);
void check_pg_backupapi_standby_clone_options(void);
#endif /* _REPMGR_ACTION_STANDBY_H_ */ #endif /* _REPMGR_ACTION_STANDBY_H_ */