Compare commits

..

9 Commits
v3.2 ... v3.1.1

Author SHA1 Message Date
Ian Barwick
5ad674edff Bump version
3.1.1
2016-02-23 15:56:24 +09:00
Ian Barwick
ac09bad89c Minor fixes to README.md 2016-02-23 14:37:59 +09:00
Ian Barwick
009d92fec8 Ensure witness node is registered before the repl_nodes table is copied
This fixes a bug introduced into the previous commit, where the
witness node was registered last to prevent a spurious node record
being created even if witness server creation failed.
2016-02-23 14:37:54 +09:00
Martin
b3d8a68a1d Fix a few paragraphs from the README.md. 2016-02-23 14:37:48 +09:00
Ian Barwick
05b47cb2a8 Prevent repmgr/repmgrd running as root 2016-02-23 14:37:44 +09:00
Ian Barwick
dc542a1b7d Better handling of errors during witness creation
Ensure witness is only registered after all steps for creation
have been successfully completed.

Also write an event record if connection could not be made to
the witness server after initial creation.

This addresses GitHub issue #146.
2016-02-23 14:37:39 +09:00
Ian Barwick
6ce8058749 witness creation: extract database and user names from the local conninfo string
99.9% of the time they'll be the same as the primary connection, but
it's more consistent to use the provided local conninfo string
(from which the port is already extracted).
2016-02-23 14:37:31 +09:00
Ian Barwick
2edcac77f0 README.md: update witness server section 2016-02-23 14:37:27 +09:00
Ian Barwick
f740374392 Add '-P/--pwprompt' option for "repmgr create witness"
Optionally prompt for superuser and repmgr user when creating a witness.
This ensures a password can be provided if the primary's pg_hba.conf
mandates it.

This deprecates '--initdb-no-pwprompt'; and changes the default behaviour of
"repmgr create witness", which previously required a superuser password
unless '--initdb-no-pwprompt' was supplied.

This behaviour is more consistent with other PostgreSQL utilities such
as createuser.

Partial fix for GitHub issue #145.
2016-02-23 14:37:23 +09:00
31 changed files with 1647 additions and 6204 deletions

14
FAQ.md
View File

@@ -38,7 +38,7 @@ General
No. Hash indexes and replication do not mix well and their use is
explicitly discouraged; see:
https://www.postgresql.org/docs/current/interactive/sql-createindex.html#AEN74175
http://www.postgresql.org/docs/current/interactive/sql-createindex.html#AEN74175
`repmgr`
--------
@@ -137,7 +137,6 @@ General
of events which includes servers removed from the replication cluster
which no longer have an entry in the `repl_nodes` table.
`repmgrd`
---------
@@ -152,9 +151,6 @@ General
In `repmgr.conf`, set its priority to a value of 0 or less.
Additionally, if `failover` is set to `manual`, the node will never
be considered as a promotion candidate.
- Does `repmgrd` support delayed standbys?
`repmgrd` can monitor delayed standbys - those set up with
@@ -173,11 +169,3 @@ General
Configure your system's `logrotate` service to do this; see example
in README.md
- I've recloned a failed master as a standby, but `repmgrd` refuses to start?
Check you registered the standby after recloning. If unregistered the standby
cannot be considered as a promotion candidate even if `failover` is set to
`automatic`, which is probably not what you want. `repmgrd` will start if
`failover` is set to `manual` so the node's replication status can still
be monitored, if desired.

71
HISTORY
View File

@@ -1,73 +1,4 @@
3.2 2016-10-05
repmgr: add support for cloning from a Barman backup (Gianni)
repmgr: add commands `standby matrix` and `standby crosscheck` (Gianni)
repmgr: suppress connection error display in `repmgr cluster show`
unless `--verbose` supplied (Ian)
repmgr: add commands `witness register` and `witness unregister` (Ian)
repmgr: enable `standby unregister` / `witness unregister` to be
executed for a node which is not running (Ian)
repmgr: remove deprecated command line options --initdb-no-pwprompt and
-l/--local-port (Ian)
repmgr: before cloning with pg_basebackup, check that sufficient free
walsenders are available (Ian)
repmgr: add option `--wait-sync` for `standby register` which causes
repmgr to wait for the registered node record to synchronise to
the standby (Ian)
repmgr: add option `--copy-external-config-files` for files outside
of the data directory (Ian)
repmgr: add configuration options to override the default pg_ctl
commands (Jarkko Oranen, Ian)
repmgr: only require `wal_keep_segments` to be set in certain corner
cases (Ian)
repmgr: better support cloning from a node other than the one to
stream from (Ian)
repmgrd: don't start if node is inactive and failover=automatic (Ian)
packaging: improve "repmgr-auto" Debian package (Gianni)
3.1.5 2016-08-15
repmgrd: in a failover situation, prevent endless looping when
attempting to establish the status of a node with
`failover=manual` (Ian)
repmgrd: improve handling of failover events on standbys with
`failover=manual`, and create a new event notification
for this, `standby_disconnect_manual` (Ian)
repmgr: add further event notifications (Gianni)
repmgr: when executing `standby switchover`, don't collect remote
command output unless required (Gianni, Ian)
repmgrd: improve standby monitoring query (Ian, based on suggestion
from Álvaro)
repmgr: various command line handling improvements (Ian)
3.1.4 2016-07-12
repmgr: new configuration option for setting "restore_command"
in the recovery.conf file generated by repmgr (Martín)
repmgr: add --csv option to "repmgr cluster show" (Gianni)
repmgr: enable provision of a conninfo string as the -d/--dbname
parameter, similar to other PostgreSQL utilities (Ian)
repmgr: during switchover operations improve detection of
demotion candidate shutdown (Ian)
various bugfixes and documentation updates (Ian, Martín)
3.1.3 2016-05-17
repmgrd: enable monitoring when a standby is catching up by
replaying archived WAL (Ian)
repmgrd: when upstream_node_id is NULL, assume upstream node
to be current master (Ian)
repmgrd: check for reappearance of the master node if standby
promotion fails (Ian)
improve handling of rsync failure conditions (Martín)
3.1.2 2016-04-12
Fix pg_ctl path generation in do_standby_switchover() (Ian)
Regularly sync witness server repl_nodes table (Ian)
Documentation improvements (Gianni, dhyannataraj)
(Experimental) ensure repmgr handles failover slots when copying
in rsync mode (Craig, Ian)
rsync mode handling fixes (Martín)
Enable repmgr to compile against 9.6devel (Ian)
3.1.1 2016-02-24
3.1.1 2016-02-
Add '-P/--pwprompt' option for "repmgr create witness" (Ian)
Prevent repmgr/repmgrd running as root (Ian)

View File

@@ -2,32 +2,23 @@
# Makefile
# Copyright (c) 2ndQuadrant, 2010-2016
HEADERS = $(wildcard *.h)
repmgrd_OBJS = dbutils.o config.o repmgrd.o log.o strutil.o
repmgr_OBJS = dbutils.o check_dir.o config.o repmgr.o log.o strutil.o dirmod.o
repmgr_OBJS = dbutils.o check_dir.o config.o repmgr.o log.o strutil.o
DATA = repmgr.sql uninstall_repmgr.sql
PG_CPPFLAGS = -I$(libpq_srcdir)
PG_LIBS = $(libpq_pgport)
PG_LIBS = $(libpq_pgport)
all: repmgrd repmgr
all: repmgrd repmgr
$(MAKE) -C sql
repmgrd: $(repmgrd_OBJS)
$(CC) -o repmgrd $(CFLAGS) $(repmgrd_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS)
$(CC) $(CFLAGS) $(repmgrd_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o repmgrd
$(MAKE) -C sql
repmgr: $(repmgr_OBJS)
$(CC) -o repmgr $(CFLAGS) $(repmgr_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS)
# Make all objects depend on all include files. This is a bit of a
# shotgun approach, but the codebase is small enough that a complete rebuild
# is very fast anyway.
$(repmgr_OBJS): $(HEADERS)
$(repmgrd_OBJS): $(HEADERS)
$(CC) $(CFLAGS) $(repmgr_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o repmgr
ifdef USE_PGXS
PG_CONFIG = pg_config
@@ -40,8 +31,8 @@ include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
# XXX: This overrides the pgxs install target - we're building two binaries,
# which is not supported by pgxs.mk's PROGRAM construct.
# XXX: Try to use PROGRAM construct (see pgxs.mk) someday. Right now
# is overriding pgxs install.
install: install_prog install_ext
install_prog:
@@ -52,12 +43,6 @@ install_prog:
install_ext:
$(MAKE) -C sql install
# Distribution-specific package building targets
# ----------------------------------------------
#
# XXX we recommend using the PGDG-supplied packages where possible;
# see README.md for details.
install_rhel:
mkdir -p '$(DESTDIR)/etc/init.d/'
$(INSTALL_PROGRAM) RHEL/repmgrd.init '$(DESTDIR)/etc/init.d/repmgrd'
@@ -87,12 +72,10 @@ PG_VERSION = $(shell pg_config --version | cut -d ' ' -f 2 | cut -d '.' -f 1,2)
REPMGR_VERSION = $(shell grep REPMGR_VERSION version.h | cut -d ' ' -f 3 | cut -d '"' -f 2)
PKGLIBDIR = $(shell pg_config --pkglibdir)
SHAREDIR = $(shell pg_config --sharedir)
PGBINDIR = /usr/lib/postgresql/$(PG_VERSION)/bin
deb: repmgrd repmgr
mkdir -p ./debian/usr/bin ./debian$(PGBINDIR)
cp repmgrd repmgr ./debian$(PGBINDIR)
ln -s ../..$(PGBINDIR)/repmgr ./debian/usr/bin/repmgr
mkdir -p ./debian/usr/bin
cp repmgrd repmgr ./debian/usr/bin/
mkdir -p ./debian$(SHAREDIR)/contrib/
cp sql/repmgr_funcs.sql ./debian$(SHAREDIR)/contrib/
cp sql/uninstall_repmgr_funcs.sql ./debian$(SHAREDIR)/contrib/

646
README.md
View File

@@ -7,8 +7,6 @@ replication capabilities with utilities to set up standby servers, monitor
replication, and perform administrative tasks such as failover or switchover
operations.
The current `repmgr` version, 3.2, supports all PostgreSQL versions from
9.3, including the upcoming 9.6.
Overview
--------
@@ -50,7 +48,7 @@ This guide assumes that you are familiar with PostgreSQL administration and
streaming replication concepts. For further details on streaming
replication, see this link:
https://www.postgresql.org/docs/current/interactive/warm-standby.html#STREAMING-REPLICATION
http://www.postgresql.org/docs/current/interactive/warm-standby.html#STREAMING-REPLICATION
The following terms are used throughout the `repmgr` documentation.
@@ -145,27 +143,10 @@ The `repmgr` tools must be installed on each server in the replication cluster.
A dedicated system user for `repmgr` is *not* required; as many `repmgr` and
`repmgrd` actions require direct access to the PostgreSQL data directory,
these commands should be executed by the `postgres` user.
it should be executed by the `postgres` user.
Passwordless `ssh` connectivity between all servers in the replication cluster
is not required, but is necessary in the following cases:
* if you need `repmgr` to copy configuration files from outside the PostgreSQL
data directory
* when using `rsync` to clone a standby
* to perform switchover operations
* when executing `repmgr cluster matrix` and `repmgr cluster crosscheck`
In these cases `rsync` is required on all servers too.
* * *
> *TIP*: We recommend using a session multiplexer utility such as `screen` or
> `tmux` when performing long-running actions (such as cloning a database)
> on a remote server - this will ensure the `repmgr` action won't be prematurely
> terminated if your `ssh` session to the server is interrupted or closed.
* * *
Additionally, we recommend installing `rsync` and enabling passwordless
`ssh` connectivity between all servers in the replication cluster.
### Packages
@@ -174,15 +155,9 @@ system.
- RedHat/CentOS: RPM packages for `repmgr` are available via Yum through
the PostgreSQL Global Development Group RPM repository ( http://yum.postgresql.org/ ).
Follow the instructions for your distribution (RedHat, CentOS,
You need to follow the instructions for your distribution (RedHat, CentOS,
Fedora, etc.) and architecture as detailed at yum.postgresql.org.
2ndQuadrant also provides its own RPM packages which are made available
at the same time as each `repmgr` release, as it can take some days for
them to become available via the main PGDG repository. See here for details:
http://repmgr.org/yum-repository.html
- Debian/Ubuntu: the most recent `repmgr` packages are available from the
PostgreSQL Community APT repository ( http://apt.postgresql.org/ ).
Instructions can be found in the APT section of the PostgreSQL Wiki
@@ -240,34 +215,6 @@ command line options:
- `-b/--pg_bindir`
### Command line options and environment variables
For some commands, e.g. `repmgr standby clone`, database connection parameters
need to be provided. Like other PostgreSQL utilities, following standard
parameters can be used:
- `-d/--dbname=DBNAME`
- `-h/--host=HOSTNAME`
- `-p/--port=PORT`
- `-U/--username=USERNAME`
If `-d/--dbname` contains an `=` sign or starts with a valid URI prefix (`postgresql://`
or `postgres://`), it is treated as a conninfo string. See the PostgreSQL
documentation for further details:
https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
Note that if a `conninfo` string is provided, values set in this will override any
provided as individual parameters. For example, with `-d 'host=foo' --host bar`, `foo`
will be chosen over `bar`.
Like other PostgreSQL utilities, `repmgr` will default to any values set in environment
variables if explicit command line parameters are not provided. See the PostgreSQL
documentation for further details:
https://www.postgresql.org/docs/current/static/libpq-envars.html
Setting up a simple replication cluster with repmgr
---------------------------------------------------
@@ -288,43 +235,30 @@ both servers.
### PostgreSQL configuration
On the master server, a PostgreSQL instance must be initialised and running.
The following replication settings may need to be adjusted:
# Enable replication connections; set this figure to at least one more
# than the number of standbys which will connect to this server
# (note that repmgr will execute `pg_basebackup` in WAL streaming mode,
# which requires two free WAL senders)
max_wal_senders = 10
The following replication settings must be included in `postgresql.conf`:
# Ensure WAL files contain enough information to enable read-only queries
# on the standby
wal_level = 'hot_standby'
# Enable up to 10 replication connections
max_wal_senders = 10
# How much WAL to retain on the master to allow a temporarily
# disconnected standby to catch up again. The larger this is, the
# longer the standby can be disconnected. This is needed only in
# 9.3; from 9.4, replication slots can be used instead (see below).
wal_keep_segments = 5000
# Enable read-only queries on a standby
# (Note: this will be ignored on a master but we recommend including
# it anyway)
hot_standby = on
# Enable WAL file archiving
archive_mode = on
# Set archive command to a script or application that will safely store
# you WALs in a secure place. /bin/true is an example of a command that
# ignores archiving. Use something more sensible.
archive_command = '/bin/true'
# If cloning using rsync, or you have configured `pg_basebackup_options`
# in `repmgr.conf` to include the setting `--xlog-method=fetch`, *and*
# you have not set `restore_command` in `repmgr.conf`to fetch WAL files
# from another source such as Barman, you'll need to set `wal_keep_segments`
# to a high enough value to ensure that all WAL files generated while
# the standby is being cloned are retained until the standby starts up.
# wal_keep_segments = 5000
* * *
@@ -350,11 +284,11 @@ similar to the following:
local replication repmgr trust
host replication repmgr 127.0.0.1/32 trust
host replication repmgr 192.168.1.0/24 trust
host replication repmgr 192.168.1.0/32 trust
local repmgr repmgr trust
host repmgr repmgr 127.0.0.1/32 trust
host repmgr repmgr 192.168.1.0/24 trust
host repmgr repmgr 192.168.1.0/32 trust
Adjust according to your network environment and authentication requirements.
@@ -395,16 +329,6 @@ to include this schema name, e.g.
ALTER USER repmgr SET search_path TO repmgr_test, "$user", public;
* * *
> *TIP*: for Debian-based distributions we recommend explictly setting
> `pg_bindir` to the directory where `pg_ctl` and other binaries not in
> the standard path are located. For PostgreSQL 9.5 this would be
> `/usr/lib/postgresql/9.5/bin/`.
* * *
### Initialise the master server
To enable `repmgr` to support a replication cluster, the master node must
@@ -447,59 +371,14 @@ Clone the standby with:
[2016-01-07 17:21:28] [NOTICE] you can now start your PostgreSQL server
[2016-01-07 17:21:28] [HINT] for example : pg_ctl -D /path/to/node2/data/ start
This will clone the PostgreSQL data directory files from the master at `repmgr_node1`
using PostgreSQL's `pg_basebackup` utility. A `recovery.conf` file containing the
This will clone the PostgreSQL data directory files from the master at repmgr_node1
using PostgreSQL's pg_basebackup utility. A `recovery.conf` file containing the
correct parameters to start streaming from this master server will be created
automatically.
automatically, and unless otherwise the `postgresql.conf` and `pg_hba.conf`
files will be copied from the master.
Note that by default, any configuration files in the master's data directory will be
copied to the standby. Typically these will be `postgresql.conf`, `postgresql.auto.conf`,
`pg_hba.conf` and `pg_ident.conf`. These may require modification before the standby
is started so it functions as desired.
In some cases (e.g. on Debian or Ubuntu Linux installations), PostgreSQL's
configuration files are located outside of the data directory and will
not be copied by default. `repmgr` can copy these files, either to the same
location on the standby server (provided appropriate directory and file permissions
are available), or into the standby's data directory. This requires passwordless
SSH access to the master server. Add the option `--copy-external-config-files`
to the `repmgr standby clone` command; by default files will be copied to
the same path as on the upstream server. To have them placed in the standby's
data directory, specify `--copy-external-config-files=pgdata`, but note that
any include directives in the copied files may need to be updated.
*Caveat*: when copying external configuration files: `repmgr` will only be able
to detect files which contain active settings. If a file is referenced by
an include directive but is empty, only contains comments or contains
settings which have not been activated, the file will not be copied.
* * *
> *TIP*: for reliable configuration file management we recommend using a
> configuration management tool such as Ansible, Chef, Puppet or Salt.
* * *
Be aware that when initially cloning a standby, you will need to ensure
that all required WAL files remain available while the cloning is taking
place. To ensure this happens when using the default `pg_basebackup` method,
`repmgr` will set `pg_basebackup`'s `--xlog-method` parameter to `stream`,
which will ensure all WAL files generated during the cloning process are
streamed in parallel with the main backup. Note that this requires two
replication connections to be available.
To override this behaviour, in `repmgr.conf` set `pg_basebackup`'s
`--xlog-method` parameter to `fetch`:
pg_basebackup_options='--xlog-method=fetch'
and ensure that `wal_keep_segments` is set to an appropriately high value.
See the `pg_basebackup` documentation for details:
https://www.postgresql.org/docs/current/static/app-pgbasebackup.html
Make any adjustments to the standby's PostgreSQL configuration files now,
then start the server.
Make any adjustments to the PostgreSQL configuration files now, then start the
standby server.
* * *
@@ -560,102 +439,13 @@ standby's upstream server is the replication cluster master. While of limited
use in a simple master/standby replication cluster, this information is required
to effectively manage cascading replication (see below).
* * *
> *TIP*: depending on your environment and workload, it may take some time for
> the standby's node record to propagate from the master to the standby. Some
> actions (such as starting `repmgrd`) require that the standby's node record
> is present and up-to-date to function correctly - by providing the option
> `--wait-sync` to the `repmgr standby register` command, `repmgr` will wait
> until the record is synchronised before exiting. An optional timeout (in
> seconds) can be added to this option (e.g. `--wait-sync=60`).
* * *
### Using Barman to clone a standby
`repmgr standby clone` also supports Barman, the Backup and
Replication manager (http://www.pgbarman.org/), as a provider of both
base backups and WAL files.
Barman support provides the following advantages:
- the master node does not need to perform a new backup every time a
new standby is cloned;
- a standby node can be disconnected for longer periods without losing
the ability to catch up, and without causing accumulation of WAL
files on the master node;
- therefore, `repmgr` does not need to use replication slots, and the
master node does not need to set `wal_keep_segments`.
> *NOTE*: In view of the above, Barman support is incompatible with
> the `use_replication_slots` setting in `repmgr.conf`.
In order to enable Barman support for `repmgr standby clone`, you must
ensure that:
- the name of the server configured in Barman is equal to the
`cluster_name` setting in `repmgr.conf`;
- the `barman_server` setting in `repmgr.conf` is set to the SSH
hostname of the Barman server;
- the `restore_command` setting in `repmgr.conf` is configured to
use a copy of the `barman-wal-restore` script shipped with the
`barman-cli package` (see below);
- the Barman catalogue includes at least one valid backup for this
server.
> *NOTE*: Barman support is automatically enabled if `barman_server`
> is set. Normally it is a good practice to use Barman, for instance
> when fetching a base backup while cloning a standby; in any case,
> Barman mode can be disabled using the `--without-barman` command
> line option.
> *NOTE*: if you have a non-default SSH configuration on the Barman
> server, e.g. using a port other than 22, then you can set those
> parameters in a dedicated Host section in `~/.ssh/config`
> corresponding to the value of `barman_server` in `repmgr.conf`. See
> the "Host" section in `man 5 ssh_config` for more details.
`barman-wal-restore` is a Python script provided by the Barman
development team as part of the `barman-cli` package (Barman 2.0
and later; for Barman 1.x the script is provided separately as
`barman-wal-restore.py`).
`restore_command` must then be set in `repmgr.conf` as follows:
<script> <Barman hostname> <cluster_name> %f %p
For instance, suppose that we have installed Barman on the `barmansrv`
host, and that `barman-wal-restore` is located as an executable at
`/usr/bin/barman-wal-restore`; `repmgr.conf` should include the following
lines:
barman_server=barmansrv
restore_command=/usr/bin/barman-wal-restore barmansrv test %f %p
NOTE: to use a non-default Barman configuration file on the Barman server,
specify this in `repmgr.conf` with `barman_config`:
barman_config=/path/to/barman.conf
Now we can clone a standby using the Barman server:
$ repmgr -h node1 -D 9.5/main -f /etc/repmgr.conf standby clone
[2016-06-12 20:08:35] [NOTICE] destination directory '9.5/main' provided
[2016-06-12 20:08:35] [NOTICE] getting backup from Barman...
[2016-06-12 20:08:36] [NOTICE] standby clone (from Barman) complete
[2016-06-12 20:08:36] [NOTICE] you can now start your PostgreSQL server
[2016-06-12 20:08:36] [HINT] for example : pg_ctl -D 9.5/data start
[2016-06-12 20:08:36] [HINT] After starting the server, you need to register this standby with "repmgr standby register"
Advanced options for cloning a standby
--------------------------------------
The above section demonstrates the simplest possible way to clone a standby
server. Depending on your circumstances, finer-grained control over the
cloning process may be necessary.
The above section demonstrates the simplest possible way to cloneb a standby
server. Depending on your circumstances, finer-grained controlover the cloning
process may be necessary.
### pg_basebackup options when cloning a standby
@@ -668,7 +458,7 @@ so should be used with care.
Further options can be passed to the `pg_basebackup` utility via
the setting `pg_basebackup_options` in `repmgr.conf`. See the PostgreSQL
documentation for more details of available options:
https://www.postgresql.org/docs/current/static/app-pgbasebackup.html
http://www.postgresql.org/docs/current/static/app-pgbasebackup.html
### Using rsync to clone a standby
@@ -686,32 +476,20 @@ and destination server as the contents of files existing on both servers need
to be compared, meaning this method is not necessarily faster than making a
fresh clone with `pg_basebackup`.
> *NOTE*: `barman-wal-restore` supports command line switches to
> control parallelism (`--parallel=N`) and compression (`--bzip2`,
> `--gzip`).
### Controlling `primary_conninfo` in `recovery.conf`
### Dealing with PostgreSQL configuration files
The `primary_conninfo` setting in `recovery.conf` generated by `repmgr`
is generated from the following sources, in order of highest to lowest priority:
By default, `repmgr` will attempt to copy the standard configuration files
(`postgresql.conf`, `pg_hba.conf` and `pg_ident.conf`) even if they are located
outside of the data directory (though currently they will be copied
into the standby's data directory). To prevent this happening, when executing
`repmgr standby clone` provide the `--ignore-external-config-files` option.
- the upstream node's `conninfo` setting (as defined in the `repl_nodes` table)
- the connection parameters provided to `repmgr standby clone`
- PostgreSQL's standard connection defaults, including any environment variables
set on the local node.
If using `rsync` to clone a standby, additional control over which files
not to transfer is possible by configuring `rsync_options` in `repmgr.conf`,
which enables any valid `rsync` options to be passed to that command, e.g.:
To include specific connection parameters other than the standard host, port,
username and database values (e.g. `sslmode`), include these in a `conninfo`-style
string passed to `repmgr` with `-d/--dbname` (see above for details), and/or set
appropriate environment variables.
Note that PostgreSQL will always set explicit defaults for `sslmode` and
`sslcompression`.
If `application_name` is set in the standby's `conninfo` parameter in
`repmgr.conf`, this value will be appended to `primary_conninfo`, otherwise
`repmgr` will set `application_name` to the same value as the `node_name`
parameter.
rsync_options='--exclude=postgresql.local.conf'
Setting up cascading replication with repmgr
@@ -786,10 +564,6 @@ To enable `repmgr` to use replication slots, set the boolean parameter
Note that `repmgr` will fail with an error if this option is specified when
working with PostgreSQL 9.3.
Replication slots must be enabled in `postgresql.conf` by setting the parameter
`max_replication_slots` to at least the number of expected standbys (changes
to this parameter require a server restart).
When cloning a standby, `repmgr` will automatically generate an appropriate
slot name, which is stored in the `repl_nodes` table, and create the slot
on the master:
@@ -812,9 +586,21 @@ Note that a slot name will be created by default for the master but not
actually used unless the master is converted to a standby using e.g.
`repmgr standby switchover`.
Be aware that when initially cloning a standby, you will need to ensure
that all required WAL files remain available while the cloning is taking
place. If using the default `pg_basebackup` method, we recommend setting
`pg_basebackup`'s `--xlog-method` parameter to `stream` like this:
pg_basebackup_options='--xlog-method=stream'
See the `pg_basebackup` documentation for details:
http://www.postgresql.org/docs/current/static/app-pgbasebackup.html
Otherwise it's necessary to set `wal_keep_segments` to an appropriately high
value.
Further information on replication slots in the PostgreSQL documentation:
https://www.postgresql.org/docs/current/interactive/warm-standby.html#STREAMING-REPLICATION-SLOTS
http://www.postgresql.org/docs/current/interactive/warm-standby.html#STREAMING-REPLICATION-SLOTS
Promoting a standby server with repmgr
@@ -913,9 +699,8 @@ updated to reflect this:
Note that with cascading replication, `repmgr standby follow` can also be
used to detach a standby from its current upstream server and follow the
master. However it's currently not possible to have it follow another standby;
we hope to improve this in a future release.
used to detach a standby from its current upstream server and follow another
upstream server, including the master.
Performing a switchover with repmgr
@@ -942,7 +727,7 @@ both passwordless SSH access and the path of `repmgr.conf` on that server.
> careful preparation and with adequate attention. In particular you should
> be confident that your network environment is stable and reliable.
>
> We recommend running `repmgr standby switchover` at the most verbose
> We recommend running `repmgr standby switchover` at the most verbose
> logging level (`--log-level DEBUG --verbose`) and capturing all output
> to assist troubleshooting any problems.
>
@@ -1008,25 +793,20 @@ should have been updated to reflect this:
### Caveats
- The functionality provided `repmgr standby switchover` is primarily aimed
- the functionality provided `repmgr standby switchover` is primarily aimed
at a two-server master/standby replication cluster and currently does
not support additional standbys.
- `repmgr standby switchover` is designed to use the `pg_rewind` utility,
standard in 9.5 and later and available separately in 9.3 and 9.4
standard in 9.5 and later and available for seperately in 9.3 and 9.4
(see note below)
- `pg_rewind` *requires* that either `wal_log_hints` is enabled, or that
data checksums were enabled when the cluster was initialized. See the
`pg_rewind` documentation for details:
https://www.postgresql.org/docs/current/static/app-pgrewind.html
http://www.postgresql.org/docs/current/static/app-pgrewind.html
- `repmgrd` should not be running when a switchover is carried out, otherwise
the `repmgrd` may try and promote a standby by itself.
- Any other standbys attached to the old master will need to be manually
instructed to point to the new master (e.g. with `repmgr standby follow`).
- You must ensure that following a server start using `pg_ctl`, log output
is not send to STDERR (the default behaviour). If logging is not configured,
we recommend setting `logging_collector=on` in `postgresql.conf` and
providing an explicit `-l/--log` setting in `repmgr.conf`'s `pg_ctl_options`
parameter.
We hope to remove some of these restrictions in future versions of `repmgr`.
@@ -1040,7 +820,7 @@ will have diverged slightly following the shutdown of the old master.
The utility `pg_rewind` provides an efficient way of doing this, however
is not included in the core PostgreSQL distribution for versions 9.3 and 9.4.
However, `pg_rewind` is available separately for these versions and we
Hoever, `pg_rewind` is available separately for these versions and we
strongly recommend its installation. To use it with versions 9.3 and 9.4,
provide the command line option `--pg_rewind`, optionally with the
path to the `pg_rewind` binary location if not installed in the PostgreSQL
@@ -1049,10 +829,6 @@ path to the `pg_rewind` binary location if not installed in the PostgreSQL
`pg_rewind` for versions 9.3 and 9.4 can be obtained from:
https://github.com/vmware/pg_rewind
Note that building this version of `pg_rewind` requires the PostgreSQL source
code. Also, PostgreSQL 9.3 does not provide `wal_log_hints`, meaning data
checksums must have been enabled when the database was initialized.
If `pg_rewind` is not available, as a fallback `repmgr` will use `repmgr
standby clone` to resynchronise the old master's data directory using
`rsync`. However, in order to ensure all files are synchronised, the
@@ -1075,17 +851,17 @@ recorded in the `repl_events` table.
Note that this command will not stop the server itself or remove
it from the replication cluster.
If the standby is not running, the command can be executed on another
node by providing the id of the node to be unregistered using
the command line parameter `--node`, e.g. executing the following
command on the master server will unregister the standby with
id 3:
If the standby is not running, the standby record must be manually
removed from the `repl_nodes` table with e.g.:
repmgr standby unregister -f /etc/repmgr.conf --node=3
DELETE FROM repmgr_test.repl_nodes WHERE id = 3;
Adjust schema and node ID accordingly. A future `repmgr` release
will make it possible to unregister failed standbys.
Automatic failover with `repmgrd`
---------------------------------
Automatic failover with repmgrd
-------------------------------
`repmgrd` is a management and monitoring daemon which runs on standby nodes
and which can automate actions such as failover and updating standbys to
@@ -1095,15 +871,11 @@ To use `repmgrd` for automatic failover, the following `repmgrd` options must
be set in `repmgr.conf`:
failover=automatic
promote_command='repmgr standby promote -f /etc/repmgr.conf'
follow_command='repmgr standby follow -f /etc/repmgr.conf'
promote_command='repmgr standby promote -f /etc/repmgr/repmgr.conf'
follow_command='repmgr standby follow -f /etc/repmgr/repmgr.conf'
(See `repmgr.conf.sample` for further `repmgrd`-specific settings).
Additionally, `postgresql.conf` must contain the following line:
shared_preload_libraries = 'repmgr_funcs'
When `failover` is set to `automatic`, upon detecting failure of the current
master, `repmgrd` will execute one of `promote_command` or `follow_command`,
depending on whether the current server is becoming the new master or
@@ -1113,7 +885,7 @@ actions happening, but we strongly recommend executing `repmgr` directly.
`repmgrd` can be started simply with e.g.:
repmgrd -f /etc/repmgr.conf --verbose >> $HOME/repmgr/repmgr.log 2>&1
repmgrd -f /etc/repmgr.conf --verbose > $HOME/repmgr/repmgr.log 2>&1
For permanent operation, we recommend using the options `-d/--daemonize` to
detach the `repmgrd` process, and `-p/--pid-file` to write the process PID
@@ -1135,7 +907,7 @@ table looks like this:
Start `repmgrd` on each standby and verify that it's running by examining
the log output, which at log level INFO will look like this:
the log output, which at default log level will look like this:
[2016-01-05 13:15:40] [INFO] checking cluster configuration with schema 'repmgr_test'
[2016-01-05 13:15:40] [INFO] checking node 2 in cluster 'test'
@@ -1205,8 +977,8 @@ during the failover:
(3 rows)
`repmgrd` log rotation
----------------------
repmgrd log rotation
--------------------
Note that currently `repmgrd` does not provide logfile rotation. To ensure
the current logfile does not grow indefinitely, configure your system's `logrotate`
@@ -1222,36 +994,12 @@ for up to 52 weeks and rotation forced if a file grows beyond 100Mb:
create 0600 postgres postgres
}
`repmgrd` and PostgreSQL connection settings
--------------------------------------------
In addition to the `repmgr` configuration settings, parameters in the
`conninfo` string influence how `repmgr` makes a network connection to
PostgreSQL. In particular, if another server in the replication cluster
is unreachable at network level, system network settings will influence
the length of time it takes to determine that the connection is not possible.
In particular explicitly setting a parameter for `connect_timeout` should
be considered; the effective minimum value of `2` (seconds) will ensure
that a connection failure at network level is reported as soon as possible,
otherwise depending on the system settings (e.g. `tcp_syn_retries` in Linux)
a delay of a minute or more is possible.
For further details on `conninfo` network connection parameters, see:
https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS
Monitoring with `repmgrd`
-------------------------
Monitoring
----------
When `repmgrd` is running with the option `-m/--monitoring-history`, it will
constantly write standby node status information to the `repl_monitor` table,
providing a near-real time overview of replication status on all nodes
in the cluster.
The view `repl_status` shows the most recent state for each node, e.g.:
constantly write node status information to the `repl_monitor` table, which can
be queried easily using the view `repl_status`:
repmgr=# SELECT * FROM repmgr_test.repl_status;
-[ RECORD 1 ]-------------+-----------------------------
@@ -1276,17 +1024,6 @@ table , it's advisable to regularly purge historical data with
`repmgr cluster cleanup`; use the `-k/--keep-history` to specify how
many day's worth of data should be retained.
It's possible to use `repmgrd` to provide monitoring only for some or all
nodes by setting `failover = manual` in the node's `repmgr.conf`. In the
event of the node's upstream failing, no failover action will be taken
and the node will require manual intervention to be reattached to replication.
If this occurs, event notification `standby_disconnect_manual` will be
created.
Note that when a standby node is not streaming directly from its upstream
node, e.g. recovering WAL from an archive, `apply_lag` will always appear as
`0 bytes`.
Using a witness server with repmgrd
------------------------------------
@@ -1382,10 +1119,8 @@ The following event types are available:
* `standby_promote`
* `standby_follow`
* `standby_switchover`
* `standby_disconnect_manual`
* `witness_create`
* `witness_register`
* `witness_unregister`
* `witness_create`
* `repmgrd_start`
* `repmgrd_shutdown`
* `repmgrd_failover_promote`
@@ -1407,42 +1142,6 @@ In general `repmgr` can be upgraded as-is without any further action required,
however feature releases may require the `repmgr` database to be upgraded.
An SQL script will be provided - please check the release notes for details.
Distribution-specific configuration
-----------------------------------
`repmgr` is largely OS-agnostic and can be run on any UNIX-like environment
including various Linux distributions, Solaris, macOS and the various BSDs.
However, often OS-specific configuration is required, particularly when
dealing with system service management (e.g. stopping and starting the
PostgreSQL server), file paths and configuration file locations.
### PostgreSQL server control
By default, `repmgr` will use PostgreSQL's standard `pg_ctl` utility to control
a running PostgreSQL server. However it may be better to use the operating
system's service management system, e.g. `systemd`. To specify which service
control commands are used, the following `repmgr.conf` configuration settings
are available:
service_start_command
service_stop_command
service_restart_command
service_reload_command
service_promote_command
See `repmgr.conf.sample` for further details.
### Binary directory
Some PostgreSQL system packages, such as those provided for Debian/Ubuntu, like
to hide some PostgreSQL utility programs outside of the default path. To ensure
`repmgr` finds all required executables, explicitly set `pg_bindir` to the
appropriate location, e.g. for PostgreSQL 9.6 on Debian/Ubuntu this would be
`/usr/lib/postgresql/9.6/bin/`.
Reference
---------
@@ -1566,156 +1265,32 @@ which contains connection details for the local database.
This command also requires the location of the witness server's data
directory to be provided (`-D/--datadir`) as well as valid connection
parameters for the master server. If not explicitly provided,
database and user names will be extracted from the `conninfo` string in
`repmgr.conf`.
parameters for the master server.
By default this command will create a superuser and a repmgr user.
The `repmgr` user name will be extracted from the `conninfo` string
in `repmgr.conf`.
* `witness register`
This will set up the witness server configuration, including the witness
server's copy of the `repmgr` meta database, on a running PostgreSQL
instance and register the witness server with the master. It requires
the same command line options as `witness create`.
* `witness unregister`
Removes the entry for a witness server from the `repl_nodes` table. This
command will not shut down the witness server or remove its data directory.
* `cluster show`
Displays information about each active node in the replication cluster. This
command polls each registered server and shows its role (`master` / `standby` /
`witness`) or `FAILED` if the node doesn't respond. It polls each server
command polls each registered server and shows its role (master / standby /
witness) or `FAILED` if the node doesn't respond. It polls each server
directly and can be run on any node in the cluster; this is also useful
when analyzing connectivity from a particular node.
This command requires a valid `repmgr.conf` file to be provided; no
additional arguments are needed.
additional arguments are required.
Example:
$ repmgr -f /etc/repmgr.conf cluster show
Role | Name | Upstream | Connection String
----------+-------|----------|----------------------------------------
* master | node1 | | host=db_node1 dbname=repmgr user=repmgr
standby | node2 | node1 | host=db_node2 dbname=repmgr user=repmgr
standby | node3 | node2 | host=db_node3 dbname=repmgr user=repmgr
To show database connection errors when polling nodes, run the command in
`--verbose` mode.
The `cluster show` command now accepts the optional parameter `--csv`, which
outputs the replication cluster's status in a simple CSV format, suitable for
parsing by scripts:
$ repmgr -f /etc/repmgr.conf cluster show --csv
1,-1
2,0
3,1
The first column is the node's ID, and the second column represents the
node's status (0 = available, -1 = failed).
* `cluster matrix` and `cluster crosscheck`
These commands display connection information for each pair of
nodes in the replication cluster.
- `cluster matrix` runs a `cluster show` on each node and arranges
the results in a matrix, recording success or failure;
- `cluster crosscheck` runs a `cluster matrix` on each node and
combines the results in a single matrix, providing a full
overview of connections between all databases in the cluster.
These commands require a valid `repmgr.conf` file on each node.
Additionally password-less `ssh` connections are required between
all nodes.
Example 1 (all nodes up):
$ repmgr -f /etc/repmgr.conf cluster matrix
Name | Id | 1 | 2 | 3
-------+----+----+----+----
node1 | 1 | * | * | *
node2 | 2 | * | * | *
node3 | 3 | * | * | *
Here `cluster matrix` is sufficient to establish the state of each
possible connection.
Example 2 (node1 and `node2` up, `node3` down):
$ repmgr -f /etc/repmgr.conf cluster matrix
Name | Id | 1 | 2 | 3
-------+----+----+----+----
node1 | 1 | * | * | x
node2 | 2 | * | * | x
node3 | 3 | ? | ? | ?
Each row corresponds to one server, and indicates the result of
testing an outbound connection from that server.
Since `node3` is down, all the entries in its row are filled with
"?", meaning that there we cannot test outbound connections.
The other two nodes are up; the corresponding rows have "x" in the
column corresponding to node3, meaning that inbound connections to
that node have failed, and "*" in the columns corresponding to
node1 and node2, meaning that inbound connections to these nodes
have succeeded.
In this case, `cluster crosscheck` gives the same result as `cluster
matrix`, because from any functioning node we can observe the same
state: `node1` and `node2` are up, `node3` is down.
Example 3 (all nodes up, firewall dropping packets originating
from `node1` and directed to port 5432 on node3)
Running `cluster matrix` from `node1` gives the following output:
$ repmgr -f /etc/repmgr.conf cluster matrix
Name | Id | 1 | 2 | 3
-------+----+----+----+----
node1 | 1 | * | * | x
node2 | 2 | * | * | *
node3 | 3 | ? | ? | ?
(Note this may take some time depending on the `connect_timeout`
setting in the registered node `conninfo` strings; default is 1
minute which means without modification the above command would
take around 2 minutes to run; see comment elsewhere about setting
`connect_timeout`)
The matrix tells us that we cannot connect from `node1` to `node3`,
and that (therefore) we don't know the state of any outbound
connection from node3.
In this case, the `cluster crosscheck` command is more informative:
$ repmgr -f /etc/repmgr.conf cluster crosscheck
Name | Id | 1 | 2 | 3
-------+----+----+----+----
node1 | 1 | * | * | x
node2 | 2 | * | * | *
node3 | 3 | * | * | *
What happened is that `cluster crosscheck` merged its own `cluster
matrix` with the `cluster matrix` output from `node2`; the latter is
able to connect to `node3` and therefore determine the state of
outbound connections from that node.
----------+-------|----------|--------------------------------------------
* master | node1 | | host=repmgr_node1 dbname=repmgr user=repmgr
standby | node2 | node1 | host=repmgr_node1 dbname=repmgr user=repmgr
standby | node3 | node2 | host=repmgr_node1 dbname=repmgr user=repmgr
* `cluster cleanup`
@@ -1729,45 +1304,25 @@ which contains connection details for the local database.
the current working directory; no additional arguments are required.
### Further documentation
As well as this README, the `repmgr` source contains following additional
documentation files:
* FAQ.md - frequently asked questions
* CONTRIBUTING.md - how to contribute to `repmgr`
* PACKAGES.md - details on building packages
* SSH-RSYNC.md - how to set up passwordless SSH between nodes
* docs/repmgrd-failover-mechanism.md - how repmgrd picks which node to promote
* docs/repmgrd-node-fencing.md - how to "fence" a failed master node
### Error codes
`repmgr` or `repmgrd` will return one of the following error codes on program
exit:
* SUCCESS (0) Program ran successfully.
* ERR_BAD_CONFIG (1) Configuration file could not be parsed or was invalid
* ERR_BAD_RSYNC (2) An rsync call made by the program returned an error
(repmgr only)
* ERR_NO_RESTART (4) An attempt to restart a PostgreSQL instance failed
* ERR_DB_CON (6) Error when trying to connect to a database
* ERR_DB_QUERY (7) Error while executing a database query
* ERR_PROMOTED (8) Exiting program because the node has been promoted to master
* ERR_STR_OVERFLOW (10) String overflow error
* ERR_FAILOVER_FAIL (11) Error encountered during failover (repmgrd only)
* ERR_BAD_SSH (12) Error when connecting to remote host via SSH (repmgr only)
* ERR_SYS_FAILURE (13) Error when forking (repmgrd only)
* ERR_BAD_BASEBACKUP (14) Error when executing pg_basebackup (repmgr only)
* ERR_MONITORING_FAIL (16) Unrecoverable error encountered during monitoring (repmgrd only)
* ERR_BAD_BACKUP_LABEL (17) Corrupt or unreadable backup label encountered (repmgr only)
* ERR_SWITCHOVER_FAIL (18) Error encountered during switchover (repmgr only)
* ERR_BARMAN (19) Unrecoverable error while accessing the barman server (repmgr only)
* ERR_REGISTRATION_SYNC (20) After registering a standby, local node record was not
syncrhonised (repmgr only, with --wait option)
* SUCCESS (0) Program ran successfully.
* ERR_BAD_CONFIG (1) Configuration file could not be parsed or was invalid
* ERR_BAD_RSYNC (2) An rsync call made by the program returned an error
* ERR_NO_RESTART (4) An attempt to restart a PostgreSQL instance failed
* ERR_DB_CON (6) Error when trying to connect to a database
* ERR_DB_QUERY (7) Error while executing a database query
* ERR_PROMOTED (8) Exiting program because the node has been promoted to master
* ERR_BAD_PASSWORD (9) Password used to connect to a database was rejected
* ERR_STR_OVERFLOW (10) String overflow error
* ERR_FAILOVER_FAIL (11) Error encountered during failover (repmgrd only)
* ERR_BAD_SSH (12) Error when connecting to remote host via SSH
* ERR_SYS_FAILURE (13) Error when forking (repmgrd only)
* ERR_BAD_BASEBACKUP (14) Error when executing pg_basebackup
* ERR_MONITORING_FAIL (16) Unrecoverable error encountered during monitoring (repmgrd only)
Support and Assistance
----------------------
@@ -1813,6 +1368,5 @@ Thanks from the repmgr core team.
Further reading
---------------
* http://blog.2ndquadrant.com/improvements-in-repmgr-3-1-4/
* http://blog.2ndquadrant.com/managing-useful-clusters-repmgr/
* http://blog.2ndquadrant.com/easier_postgresql_90_clusters/

61
RHEL/repmgr3-93.spec Normal file
View File

@@ -0,0 +1,61 @@
Summary: repmgr
Name: repmgr
Version: 3.0
Release: 1
License: GPLv3
Group: System Environment/Daemons
URL: http://repmgr.org
Packager: Ian Barwick <ian@2ndquadrant.com>
Vendor: 2ndQuadrant Limited
Distribution: centos
Source0: %{name}-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
%description
repmgr is a utility suite which greatly simplifies
the process of setting up and managing replication
using streaming replication within a cluster of
PostgreSQL servers.
%prep
%setup
%build
export PATH=$PATH:/usr/pgsql-9.3/bin/
%{__make} USE_PGXS=1
%install
[ "%{buildroot}" != "/" ] && %{__rm} -rf %{buildroot}
export PATH=$PATH:/usr/pgsql-9.3/bin/
%{__make} USE_PGXS=1 install DESTDIR=%{buildroot} INSTALL="install -p"
%{__make} USE_PGXS=1 install_prog DESTDIR=%{buildroot} INSTALL="install -p"
%{__make} USE_PGXS=1 install_rhel DESTDIR=%{buildroot} INSTALL="install -p"
%clean
[ "%{buildroot}" != "/" ] && %{__rm} -rf %{buildroot}
%files
%defattr(-,root,root)
/usr/bin/repmgr
/usr/bin/repmgrd
/usr/pgsql-9.3/bin/repmgr
/usr/pgsql-9.3/bin/repmgrd
/usr/pgsql-9.3/lib/repmgr_funcs.so
/usr/pgsql-9.3/share/contrib/repmgr.sql
/usr/pgsql-9.3/share/contrib/repmgr_funcs.sql
/usr/pgsql-9.3/share/contrib/uninstall_repmgr.sql
/usr/pgsql-9.3/share/contrib/uninstall_repmgr_funcs.sql
%attr(0755,root,root)/etc/init.d/repmgrd
%attr(0644,root,root)/etc/sysconfig/repmgrd
%attr(0644,root,root)/etc/repmgr/repmgr.conf.sample
%changelog
* Tue Mar 10 2015 Ian Barwick ian@2ndquadrant.com>
- build for repmgr 3.0
* Thu Jun 05 2014 Nathan Van Overloop <nathan.van.overloop@nexperteam.be> 2.0.2
- fix witness creation to create db and user if needed
* Fri Apr 04 2014 Nathan Van Overloop <nathan.van.overloop@nexperteam.be> 2.0.1
- initial build for RHEL6

133
RHEL/repmgrd.init Executable file
View File

@@ -0,0 +1,133 @@
#!/bin/sh
#
# chkconfig: - 75 16
# description: Enable repmgrd replication management and monitoring daemon for PostgreSQL
# processname: repmgrd
# pidfile="/var/run/${NAME}.pid"
# Source function library.
INITD=/etc/rc.d/init.d
. $INITD/functions
# Get function listing for cross-distribution logic.
TYPESET=`typeset -f|grep "declare"`
# Get network config.
. /etc/sysconfig/network
DESC="PostgreSQL replication management and monitoring daemon"
NAME=repmgrd
REPMGRD_ENABLED=no
REPMGRD_OPTS=
REPMGRD_USER=postgres
REPMGRD_BIN=/usr/pgsql-9.3/bin/repmgrd
REPMGRD_PIDFILE=/var/run/repmgrd.pid
REPMGRD_LOCK=/var/lock/subsys/${NAME}
REPMGRD_LOG=/var/lib/pgsql/9.3/data/pg_log/repmgrd.log
# Read configuration variable file if it is present
[ -r /etc/sysconfig/$NAME ] && . /etc/sysconfig/$NAME
# For SELinux we need to use 'runuser' not 'su'
if [ -x /sbin/runuser ]
then
SU=runuser
else
SU=su
fi
test -x $REPMGRD_BIN || exit 0
case "$REPMGRD_ENABLED" in
[Yy]*)
break
;;
*)
exit 0
;;
esac
if [ -z "${REPMGRD_OPTS}" ]
then
echo "Not starting ${NAME}, REPMGRD_OPTS not set in /etc/sysconfig/${NAME}"
exit 0
fi
start()
{
REPMGRD_START=$"Starting ${NAME} service: "
# Make sure startup-time log file is valid
if [ ! -e "${REPMGRD_LOG}" -a ! -h "${REPMGRD_LOG}" ]
then
touch "${REPMGRD_LOG}" || exit 1
chown ${REPMGRD_USER}:postgres "${REPMGRD_LOG}"
chmod go-rwx "${REPMGRD_LOG}"
[ -x /sbin/restorecon ] && /sbin/restorecon "${REPMGRD_LOG}"
fi
echo -n "${REPMGRD_START}"
$SU -l $REPMGRD_USER -c "${REPMGRD_BIN} ${REPMGRD_OPTS} -p ${REPMGRD_PIDFILE} &" >> "${REPMGRD_LOG}" 2>&1 < /dev/null
sleep 2
pid=`head -n 1 "${REPMGRD_PIDFILE}" 2>/dev/null`
if [ "x${pid}" != "x" ]
then
success "${REPMGRD_START}"
touch "${REPMGRD_LOCK}"
echo $pid > "${REPMGRD_PIDFILE}"
echo
else
failure "${REPMGRD_START}"
echo
script_result=1
fi
}
stop()
{
echo -n $"Stopping ${NAME} service: "
if [ -e "${REPMGRD_LOCK}" ]
then
killproc ${NAME}
ret=$?
if [ $ret -eq 0 ]
then
echo_success
rm -f "${REPMGRD_PIDFILE}"
rm -f "${REPMGRD_LOCK}"
else
echo_failure
script_result=1
fi
else
# not running; per LSB standards this is "ok"
echo_success
fi
echo
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $REPMGRD_PIDFILE $NAME
script_result=$?
;;
restart)
stop
start
;;
*)
echo $"Usage: $0 {start|stop|status|restart}"
exit 2
esac
exit $script_result

21
RHEL/repmgrd.sysconfig Normal file
View File

@@ -0,0 +1,21 @@
# default settings for repmgrd. This file is source by /bin/sh from
# /etc/init.d/repmgrd
# disable repmgrd by default so it won't get started upon installation
# valid values: yes/no
REPMGRD_ENABLED=no
# Options for repmgrd (required)
#REPMGRD_OPTS="--verbose -d -f /var/lib/pgsql/repmgr/repmgr.conf"
# User to run repmgrd as
#REPMGRD_USER=postgres
# repmgrd binary
#REPMGRD_BIN=/usr/bin/repmgrd
# pid file
#REPMGRD_PIDFILE=/var/lib/pgsql/repmgr/repmgrd.pid
# log file
#REPMGRD_LOG=/var/lib/pgsql/repmgr/repmgrd.log

5
TODO
View File

@@ -53,9 +53,8 @@ Planned feature improvements
requested, activate the replication slot using pg_receivexlog to negate the
need to set `wal_keep_segments` just for the initial clone (9.4 and 9.5).
* repmgr: enable "standby follow" to point a standby at another standby, not
just the replication cluster master (see GitHub #130)
* Take into account the fact that a standby can obtain WAL from an archive,
so even if direct streaming replication is interrupted, it may be up-to-date
Usability improvements
======================

View File

@@ -1,6 +1,5 @@
/*
* config.c - Functions to parse the config file
*
* Copyright (C) 2ndQuadrant, 2010-2016
*
* This program is free software: you can redistribute it and/or modify
@@ -27,9 +26,9 @@
static void parse_event_notifications_list(t_configuration_options *options, const char *arg);
static void tablespace_list_append(t_configuration_options *options, const char *arg);
static void exit_with_errors(ItemList *config_errors);
static void exit_with_errors(ErrorList *config_errors);
const static char *_progname = NULL;
const static char *_progname = '\0';
static char config_file_path[MAXPGPATH];
static bool config_file_provided = false;
bool config_file_found = false;
@@ -202,7 +201,7 @@ parse_config(t_configuration_options *options)
char *conninfo_errmsg = NULL;
/* Collate configuration file errors here for friendlier reporting */
static ItemList config_errors = { NULL, NULL };
static ErrorList config_errors = { NULL, NULL };
bool node_found = false;
@@ -215,24 +214,16 @@ parse_config(t_configuration_options *options)
options->upstream_node = NO_UPSTREAM_NODE;
options->use_replication_slots = 0;
memset(options->conninfo, 0, sizeof(options->conninfo));
memset(options->barman_server, 0, sizeof(options->barman_server));
memset(options->barman_config, 0, sizeof(options->barman_config));
options->failover = MANUAL_FAILOVER;
options->priority = DEFAULT_PRIORITY;
memset(options->node_name, 0, sizeof(options->node_name));
memset(options->promote_command, 0, sizeof(options->promote_command));
memset(options->follow_command, 0, sizeof(options->follow_command));
memset(options->service_stop_command, 0, sizeof(options->service_stop_command));
memset(options->service_start_command, 0, sizeof(options->service_start_command));
memset(options->service_restart_command, 0, sizeof(options->service_restart_command));
memset(options->service_reload_command, 0, sizeof(options->service_reload_command));
memset(options->service_promote_command, 0, sizeof(options->service_promote_command));
memset(options->rsync_options, 0, sizeof(options->rsync_options));
memset(options->ssh_options, 0, sizeof(options->ssh_options));
memset(options->pg_bindir, 0, sizeof(options->pg_bindir));
memset(options->pg_ctl_options, 0, sizeof(options->pg_ctl_options));
memset(options->pg_basebackup_options, 0, sizeof(options->pg_basebackup_options));
memset(options->restore_command, 0, sizeof(options->restore_command));
/* default master_response_timeout is 60 seconds */
options->master_response_timeout = 60;
@@ -244,12 +235,7 @@ parse_config(t_configuration_options *options)
options->monitor_interval_secs = 2;
options->retry_promote_interval_secs = 300;
/* default to resyncing repl_nodes table every 30 seconds on the witness server */
options->witness_repl_nodes_sync_interval_secs = 30;
memset(options->event_notification_command, 0, sizeof(options->event_notification_command));
options->event_notifications.head = NULL;
options->event_notifications.tail = NULL;
options->tablespace_mapping.head = NULL;
options->tablespace_mapping.tail = NULL;
@@ -314,10 +300,6 @@ parse_config(t_configuration_options *options)
options->upstream_node = repmgr_atoi(value, "upstream_node", &config_errors, false);
else if (strcmp(name, "conninfo") == 0)
strncpy(options->conninfo, value, MAXLEN);
else if (strcmp(name, "barman_server") == 0)
strncpy(options->barman_server, value, MAXLEN);
else if (strcmp(name, "barman_config") == 0)
strncpy(options->barman_config, value, MAXLEN);
else if (strcmp(name, "rsync_options") == 0)
strncpy(options->rsync_options, value, QUERY_STR_LEN);
else if (strcmp(name, "ssh_options") == 0)
@@ -342,7 +324,7 @@ parse_config(t_configuration_options *options)
}
else
{
item_list_append(&config_errors,_("value for 'failover' must be 'automatic' or 'manual'\n"));
error_list_append(&config_errors,_("value for 'failover' must be 'automatic' or 'manual'\n"));
}
}
else if (strcmp(name, "priority") == 0)
@@ -353,20 +335,9 @@ parse_config(t_configuration_options *options)
strncpy(options->promote_command, value, MAXLEN);
else if (strcmp(name, "follow_command") == 0)
strncpy(options->follow_command, value, MAXLEN);
else if (strcmp(name, "service_stop_command") == 0)
strncpy(options->service_stop_command, value, MAXLEN);
else if (strcmp(name, "service_start_command") == 0)
strncpy(options->service_start_command, value, MAXLEN);
else if (strcmp(name, "service_restart_command") == 0)
strncpy(options->service_restart_command, value, MAXLEN);
else if (strcmp(name, "service_reload_command") == 0)
strncpy(options->service_reload_command, value, MAXLEN);
else if (strcmp(name, "service_promote_command") == 0)
strncpy(options->service_promote_command, value, MAXLEN);
else if (strcmp(name, "master_response_timeout") == 0)
options->master_response_timeout = repmgr_atoi(value, "master_response_timeout", &config_errors, false);
/*
* 'primary_response_timeout' as synonym for 'master_response_timeout' -
/* 'primary_response_timeout' as synonym for 'master_response_timeout' -
* we'll switch terminology in a future release (3.1?)
*/
else if (strcmp(name, "primary_response_timeout") == 0)
@@ -387,8 +358,6 @@ parse_config(t_configuration_options *options)
options->monitor_interval_secs = repmgr_atoi(value, "monitor_interval_secs", &config_errors, false);
else if (strcmp(name, "retry_promote_interval_secs") == 0)
options->retry_promote_interval_secs = repmgr_atoi(value, "retry_promote_interval_secs", &config_errors, false);
else if (strcmp(name, "witness_repl_nodes_sync_interval_secs") == 0)
options->witness_repl_nodes_sync_interval_secs = repmgr_atoi(value, "witness_repl_nodes_sync_interval_secs", &config_errors, false);
else if (strcmp(name, "use_replication_slots") == 0)
/* XXX we should have a dedicated boolean argument format */
options->use_replication_slots = repmgr_atoi(value, "use_replication_slots", &config_errors, false);
@@ -398,8 +367,6 @@ parse_config(t_configuration_options *options)
parse_event_notifications_list(options, value);
else if (strcmp(name, "tablespace_mapping") == 0)
tablespace_list_append(options, value);
else if (strcmp(name, "restore_command") == 0)
strncpy(options->restore_command, value, MAXLEN);
else
{
known_parameter = false;
@@ -419,7 +386,7 @@ parse_config(t_configuration_options *options)
_("no value provided for parameter \"%s\""),
name);
item_list_append(&config_errors, error_message_buf);
error_list_append(&config_errors, error_message_buf);
}
}
@@ -428,11 +395,11 @@ parse_config(t_configuration_options *options)
if (node_found == false)
{
item_list_append(&config_errors, _("\"node\": parameter was not found"));
error_list_append(&config_errors, _("\"node\": parameter was not found"));
}
else if (options->node == 0)
{
item_list_append(&config_errors, _("\"node\": must be greater than zero"));
error_list_append(&config_errors, _("\"node\": must be greater than zero"));
}
if (strlen(options->conninfo))
@@ -452,7 +419,7 @@ parse_config(t_configuration_options *options)
_("\"conninfo\": %s"),
conninfo_errmsg);
item_list_append(&config_errors, error_message_buf);
error_list_append(&config_errors, error_message_buf);
}
PQconninfoFree(conninfo_options);
@@ -647,13 +614,6 @@ reload_config(t_configuration_options *orig_options)
config_changed = true;
}
/* barman_server */
if (strcmp(orig_options->barman_server, new_options.barman_server) != 0)
{
strcpy(orig_options->barman_server, new_options.barman_server);
config_changed = true;
}
/* node */
if (orig_options->node != new_options.node)
{
@@ -790,11 +750,11 @@ reload_config(t_configuration_options *orig_options)
void
item_list_append(ItemList *item_list, char *error_message)
error_list_append(ErrorList *error_list, char *error_message)
{
ItemListCell *cell;
ErrorListCell *cell;
cell = (ItemListCell *) pg_malloc0(sizeof(ItemListCell));
cell = (ErrorListCell *) pg_malloc0(sizeof(ErrorListCell));
if (cell == NULL)
{
@@ -802,19 +762,19 @@ item_list_append(ItemList *item_list, char *error_message)
exit(ERR_BAD_CONFIG);
}
cell->string = pg_malloc0(MAXLEN);
strncpy(cell->string, error_message, MAXLEN);
cell->error_message = pg_malloc0(MAXLEN);
strncpy(cell->error_message, error_message, MAXLEN);
if (item_list->tail)
if (error_list->tail)
{
item_list->tail->next = cell;
error_list->tail->next = cell;
}
else
{
item_list->head = cell;
error_list->head = cell;
}
item_list->tail = cell;
error_list->tail = cell;
}
@@ -824,7 +784,7 @@ item_list_append(ItemList *item_list, char *error_message)
* otherwise exit
*/
int
repmgr_atoi(const char *value, const char *config_item, ItemList *error_list, bool allow_negative)
repmgr_atoi(const char *value, const char *config_item, ErrorList *error_list, bool allow_negative)
{
char *endptr;
long longval = 0;
@@ -873,7 +833,7 @@ repmgr_atoi(const char *value, const char *config_item, ItemList *error_list, bo
exit(ERR_BAD_CONFIG);
}
item_list_append(error_list, error_message_buf);
error_list_append(error_list, error_message_buf);
}
return (int32) longval;
@@ -1015,15 +975,15 @@ parse_event_notifications_list(t_configuration_options *options, const char *arg
static void
exit_with_errors(ItemList *config_errors)
exit_with_errors(ErrorList *config_errors)
{
ItemListCell *cell;
ErrorListCell *cell;
log_err(_("%s: following errors were found in the configuration file.\n"), progname());
for (cell = config_errors->head; cell; cell = cell->next)
{
log_err("%s\n", cell->string);
log_err("%s\n", cell->error_message);
}
exit(ERR_BAD_CONFIG);

View File

@@ -1,6 +1,5 @@
/*
* config.h
*
* Copyright (c) 2ndQuadrant, 2010-2016
*
* This program is free software: you can redistribute it and/or modify
@@ -58,20 +57,11 @@ typedef struct
int node;
int upstream_node;
char conninfo[MAXLEN];
char barman_server[MAXLEN];
char barman_config[MAXLEN];
int failover;
int priority;
char node_name[MAXLEN];
/* commands executed by repmgrd */
char promote_command[MAXLEN];
char follow_command[MAXLEN];
/* Overrides for pg_ctl commands */
char service_stop_command[MAXLEN];
char service_start_command[MAXLEN];
char service_restart_command[MAXLEN];
char service_reload_command[MAXLEN];
char service_promote_command[MAXLEN];
char loglevel[MAXLEN];
char logfacility[MAXLEN];
char rsync_options[QUERY_STR_LEN];
@@ -82,50 +72,28 @@ typedef struct
char pg_bindir[MAXLEN];
char pg_ctl_options[MAXLEN];
char pg_basebackup_options[MAXLEN];
char restore_command[MAXLEN];
char logfile[MAXLEN];
int monitor_interval_secs;
int retry_promote_interval_secs;
int witness_repl_nodes_sync_interval_secs;
int use_replication_slots;
char event_notification_command[MAXLEN];
EventNotificationList event_notifications;
TablespaceList tablespace_mapping;
} t_configuration_options;
/*
* The following will initialize the structure with a minimal set of options;
* actual defaults are set in parse_config() before parsing the configuration file
*/
#define T_CONFIGURATION_OPTIONS_INITIALIZER { "", -1, NO_UPSTREAM_NODE, "", "", "", MANUAL_FAILOVER, -1, "", "", "", "", "", "", "", "", "", "", "", "", -1, -1, -1, "", "", "", "", "", 0, 0, 0, 0, "", { NULL, NULL }, { NULL, NULL } }
#define T_CONFIGURATION_OPTIONS_INITIALIZER { "", -1, NO_UPSTREAM_NODE, "", MANUAL_FAILOVER, -1, "", "", "", "", "", "", "", -1, -1, -1, "", "", "", "", 0, 0, 0, "", { NULL, NULL }, {NULL, NULL} }
typedef struct ItemListCell
typedef struct ErrorListCell
{
struct ItemListCell *next;
char *string;
} ItemListCell;
struct ErrorListCell *next;
char *error_message;
} ErrorListCell;
typedef struct ItemList
typedef struct ErrorList
{
ItemListCell *head;
ItemListCell *tail;
} ItemList;
typedef struct TablespaceDataListCell
{
struct TablespaceDataListCell *next;
char *name;
char *oid;
char *location;
/* optional payload */
FILE *f;
} TablespaceDataListCell;
typedef struct TablespaceDataList
{
TablespaceDataListCell *head;
TablespaceDataListCell *tail;
} TablespaceDataList;
ErrorListCell *head;
ErrorListCell *tail;
} ErrorList;
void set_progname(const char *argv0);
const char * progname(void);
@@ -135,10 +103,10 @@ bool reload_config(t_configuration_options *orig_options);
bool parse_config(t_configuration_options *options);
void parse_line(char *buff, char *name, char *value);
char *trim(char *s);
void item_list_append(ItemList *item_list, char *error_message);
void error_list_append(ErrorList *error_list, char *error_message);
int repmgr_atoi(const char *s,
const char *config_item,
ItemList *error_list,
ErrorList *error_list,
bool allow_negative);
extern bool config_file_found;
#endif

261
dbutils.c
View File

@@ -1,6 +1,5 @@
/*
* dbutils.c - Database connection/management functions
*
* Copyright (C) 2ndQuadrant, 2010-2016
*
* This program is free software: you can redistribute it and/or modify
@@ -32,10 +31,9 @@
char repmgr_schema[MAXLEN] = "";
char repmgr_schema_quoted[MAXLEN] = "";
static int _get_node_record(PGconn *conn, char *cluster, char *sqlquery, t_node_info *node_info);
PGconn *
_establish_db_connection(const char *conninfo, const bool exit_on_error, const bool log_notice, const bool verbose_only)
_establish_db_connection(const char *conninfo, const bool exit_on_error, const bool log_notice)
{
/* Make a connection to the database */
PGconn *conn = NULL;
@@ -51,23 +49,15 @@ _establish_db_connection(const char *conninfo, const bool exit_on_error, const b
/* Check to see that the backend connection was successfully made */
if ((PQstatus(conn) != CONNECTION_OK))
{
bool emit_log = true;
if (verbose_only == true && verbose_logging == false)
emit_log = false;
if (emit_log)
if (log_notice)
{
if (log_notice)
{
log_notice(_("connection to database failed: %s\n"),
PQerrorMessage(conn));
}
else
{
log_err(_("connection to database failed: %s\n"),
PQerrorMessage(conn));
}
log_notice(_("connection to database failed: %s\n"),
PQerrorMessage(conn));
}
else
{
log_err(_("connection to database failed: %s\n"),
PQerrorMessage(conn));
}
if (exit_on_error)
@@ -80,35 +70,16 @@ _establish_db_connection(const char *conninfo, const bool exit_on_error, const b
return conn;
}
/*
* Establish a database connection, optionally exit on error
*/
PGconn *
establish_db_connection(const char *conninfo, const bool exit_on_error)
{
return _establish_db_connection(conninfo, exit_on_error, false, false);
return _establish_db_connection(conninfo, exit_on_error, false);
}
/*
* Attempt to establish a database connection, never exit on error, only
* output error messages if --verbose option used
*/
PGconn *
establish_db_connection_quiet(const char *conninfo)
test_db_connection(const char *conninfo, const bool exit_on_error)
{
return _establish_db_connection(conninfo, false, false, true);
}
/*
* Attempt to establish a database connection, never exit on error,
* output connection error messages as NOTICE (useful when connection
* failure is expected)
*/
PGconn *
test_db_connection(const char *conninfo)
{
return _establish_db_connection(conninfo, false, true, false);
return _establish_db_connection(conninfo, exit_on_error, true);
}
@@ -214,7 +185,7 @@ check_cluster_schema(PGconn *conn)
char sqlquery[QUERY_STR_LEN];
sqlquery_snprintf(sqlquery,
"SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = '%s'",
"SELECT 1 FROM pg_namespace WHERE nspname = '%s'",
get_repmgr_schema());
log_verbose(LOG_DEBUG, "check_cluster_schema(): %s\n", sqlquery);
@@ -409,7 +380,7 @@ guc_set(PGconn *conn, const char *parameter, const char *op,
int retval = 1;
sqlquery_snprintf(sqlquery,
"SELECT true FROM pg_catalog.pg_settings "
"SELECT true FROM pg_settings "
" WHERE name = '%s' AND setting %s '%s'",
parameter, op, value);
@@ -445,11 +416,11 @@ guc_set_typed(PGconn *conn, const char *parameter, const char *op,
int retval = 1;
sqlquery_snprintf(sqlquery,
"SELECT true FROM pg_catalog.pg_settings "
"SELECT true FROM pg_settings "
" WHERE name = '%s' AND setting::%s %s '%s'::%s",
parameter, datatype, op, value, datatype);
log_verbose(LOG_DEBUG, "guc_set_typed():\n%s\n", sqlquery);
log_verbose(LOG_DEBUG, "guc_set_typed():n%s\n", sqlquery);
res = PQexec(conn, sqlquery);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
@@ -477,7 +448,7 @@ get_cluster_size(PGconn *conn, char *size)
sqlquery_snprintf(sqlquery,
"SELECT pg_catalog.pg_size_pretty(SUM(pg_catalog.pg_database_size(oid))::bigint) "
" FROM pg_catalog.pg_database ");
" FROM pg_database ");
log_verbose(LOG_DEBUG, "get_cluster_size():\n%s\n", sqlquery);
@@ -504,11 +475,11 @@ get_pg_setting(PGconn *conn, const char *setting, char *output)
char sqlquery[QUERY_STR_LEN];
PGresult *res;
int i;
bool success = false;
bool success = true;
sqlquery_snprintf(sqlquery,
"SELECT name, setting "
" FROM pg_catalog.pg_settings WHERE name = '%s'",
" FROM pg_settings WHERE name = '%s'",
setting);
log_verbose(LOG_DEBUG, "get_pg_setting(): %s\n", sqlquery);
@@ -567,7 +538,7 @@ get_conninfo_value(const char *conninfo, const char *keyword, char *output)
conninfo_options = PQconninfoParse(conninfo, NULL);
if (conninfo_options == NULL)
if (conninfo_options == false)
{
log_err(_("Unable to parse provided conninfo string \"%s\""), conninfo);
return false;
@@ -616,7 +587,7 @@ get_upstream_connection(PGconn *standby_conn, char *cluster, int node_id,
upstream_conninfo = upstream_conninfo_out;
sqlquery_snprintf(sqlquery,
" SELECT un.conninfo, un.id "
" SELECT un.conninfo, un.name, un.id "
" FROM %s.repl_nodes un "
"INNER JOIN %s.repl_nodes n "
" ON (un.id = n.upstream_node_id AND un.cluster = n.cluster)"
@@ -633,7 +604,7 @@ get_upstream_connection(PGconn *standby_conn, char *cluster, int node_id,
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
log_err(_("error when attempting to find upstream server\n%s\n"),
log_err(_("unable to get conninfo for upstream server\n%s\n"),
PQerrorMessage(standby_conn));
PQclear(res);
return NULL;
@@ -641,36 +612,9 @@ get_upstream_connection(PGconn *standby_conn, char *cluster, int node_id,
if (!PQntuples(res))
{
log_notice(_("no record found for upstream server"));
PQclear(res);
log_debug("no record found for upstream server\n");
sqlquery_snprintf(sqlquery,
" SELECT un.conninfo, un.id "
" FROM %s.repl_nodes un "
" WHERE un.cluster = '%s' "
" AND un.type='master' "
" AND un.active IS TRUE",
get_repmgr_schema_quoted(standby_conn),
cluster);
res = PQexec(standby_conn, sqlquery);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
log_err(_("error when attempting to find active master server\n%s\n"),
PQerrorMessage(standby_conn));
PQclear(res);
return NULL;
}
if (!PQntuples(res))
{
PQclear(res);
log_notice(_("no record found for active master server\n"));
return NULL;
}
log_debug("record found for active master server\n");
return NULL;
}
strncpy(upstream_conninfo, PQgetvalue(res, 0, 0), MAXCONNINFO);
@@ -945,7 +889,7 @@ get_repmgr_schema_quoted(PGconn *conn)
bool
create_replication_slot(PGconn *conn, char *slot_name, int server_version_num, PQExpBufferData *error_msg)
create_replication_slot(PGconn *conn, char *slot_name)
{
char sqlquery[QUERY_STR_LEN];
int query_res;
@@ -964,9 +908,8 @@ create_replication_slot(PGconn *conn, char *slot_name, int server_version_num, P
{
if (strcmp(slot_info.slot_type, "physical") != 0)
{
appendPQExpBuffer(error_msg,
_("Slot '%s' exists and is not a physical slot\n"),
slot_name);
log_err(_("Slot '%s' exists and is not a physical slot\n"),
slot_name);
return false;
}
@@ -978,36 +921,24 @@ create_replication_slot(PGconn *conn, char *slot_name, int server_version_num, P
return true;
}
appendPQExpBuffer(error_msg,
_("Slot '%s' already exists as an active slot\n"),
slot_name);
log_err(_("Slot '%s' already exists as an active slot\n"),
slot_name);
return false;
}
/* In 9.6 and later, reserve the LSN straight away */
if (server_version_num >= 90600)
{
sqlquery_snprintf(sqlquery,
"SELECT * FROM pg_catalog.pg_create_physical_replication_slot('%s', TRUE)",
slot_name);
}
else
{
sqlquery_snprintf(sqlquery,
"SELECT * FROM pg_catalog.pg_create_physical_replication_slot('%s')",
slot_name);
}
sqlquery_snprintf(sqlquery,
"SELECT * FROM pg_create_physical_replication_slot('%s')",
slot_name);
log_debug(_("create_replication_slot(): Creating slot '%s' on master\n"), slot_name);
log_debug(_("create_replication_slot(): Creating slot '%s' on primary\n"), slot_name);
log_verbose(LOG_DEBUG, "create_replication_slot():\n%s\n", sqlquery);
res = PQexec(conn, sqlquery);
if (!res || PQresultStatus(res) != PGRES_TUPLES_OK)
{
appendPQExpBuffer(error_msg,
_("unable to create slot '%s' on the master node: %s\n"),
slot_name,
PQerrorMessage(conn));
log_err(_("unable to create slot '%s' on the primary node: %s\n"),
slot_name,
PQerrorMessage(conn));
PQclear(res);
return false;
}
@@ -1025,7 +956,7 @@ get_slot_record(PGconn *conn, char *slot_name, t_replication_slot *record)
sqlquery_snprintf(sqlquery,
"SELECT slot_name, slot_type, active "
" FROM pg_catalog.pg_replication_slots "
" FROM pg_replication_slots "
" WHERE slot_name = '%s' ",
slot_name);
@@ -1180,7 +1111,7 @@ set_config_bool(PGconn *conn, const char *config_param, bool state)
/*
* witness_copy_node_records()
* copy_configuration()
*
* Copy records in master's `repl_nodes` table to witness database
*
@@ -1188,50 +1119,29 @@ set_config_bool(PGconn *conn, const char *config_param, bool state)
* `repmgrd` after a failover event occurs
*/
bool
witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster_name)
copy_configuration(PGconn *masterconn, PGconn *witnessconn, char *cluster_name)
{
char sqlquery[MAXLEN];
PGresult *res;
int i;
begin_transaction(witnessconn);
/* Defer constraints */
sqlquery_snprintf(sqlquery, "SET CONSTRAINTS ALL DEFERRED;");
log_verbose(LOG_DEBUG, "witness_copy_node_records():\n%s\n", sqlquery);
res = PQexec(witnessconn, sqlquery);
if (!res || PQresultStatus(res) != PGRES_COMMAND_OK)
{
log_err(_("Unable to defer constraints:\n%s\n"),
PQerrorMessage(witnessconn));
rollback_transaction(witnessconn);
return false;
}
/* Truncate existing records */
sqlquery_snprintf(sqlquery, "TRUNCATE TABLE %s.repl_nodes", get_repmgr_schema_quoted(witnessconn));
log_verbose(LOG_DEBUG, "witness_copy_node_records():\n%s\n", sqlquery);
log_verbose(LOG_DEBUG, "copy_configuration():\n%s\n", sqlquery);
res = PQexec(witnessconn, sqlquery);
if (!res || PQresultStatus(res) != PGRES_COMMAND_OK)
{
log_err(_("Unable to truncate witness servers's repl_nodes table:\n%s\n"),
PQerrorMessage(witnessconn));
rollback_transaction(witnessconn);
return false;
}
/* Get current records from primary */
sqlquery_snprintf(sqlquery,
"SELECT id, type, upstream_node_id, name, conninfo, priority, slot_name, active "
" FROM %s.repl_nodes",
"SELECT id, type, upstream_node_id, name, conninfo, priority, slot_name FROM %s.repl_nodes",
get_repmgr_schema_quoted(masterconn));
log_verbose(LOG_DEBUG, "witness_copy_node_records():\n%s\n", sqlquery);
log_verbose(LOG_DEBUG, "copy_configuration():\n%s\n", sqlquery);
res = PQexec(masterconn, sqlquery);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
@@ -1239,23 +1149,20 @@ witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster
log_err("Unable to retrieve node records from master:\n%s\n",
PQerrorMessage(masterconn));
PQclear(res);
rollback_transaction(witnessconn);
return false;
}
/* Insert primary records into witness table */
for (i = 0; i < PQntuples(res); i++)
{
bool node_record_created;
log_verbose(LOG_DEBUG,
"witness_copy_node_records(): writing node record for node %s (id: %s)\n",
PQgetvalue(res, i, 3),
"copy_configuration(): writing node record for node %s (id: %s)\n",
PQgetvalue(res, i, 4),
PQgetvalue(res, i, 0));
node_record_created = create_node_record(witnessconn,
"witness_copy_node_records",
"copy_configuration",
atoi(PQgetvalue(res, i, 0)),
PQgetvalue(res, i, 1),
strlen(PQgetvalue(res, i, 2))
@@ -1267,10 +1174,7 @@ witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster
atoi(PQgetvalue(res, i, 5)),
strlen(PQgetvalue(res, i, 6))
? PQgetvalue(res, i, 6)
: NULL,
(strcmp(PQgetvalue(res, i, 7), "t") == 0)
? true
: false
: NULL
);
if (node_record_created == false)
@@ -1279,16 +1183,11 @@ witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster
log_err("Unable to copy node record to witness database\n%s\n",
PQerrorMessage(witnessconn));
rollback_transaction(witnessconn);
return false;
}
}
PQclear(res);
/* And finished */
commit_transaction(witnessconn);
return true;
}
@@ -1301,7 +1200,7 @@ witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster
* XXX we should pass the record parameters as a struct.
*/
bool
create_node_record(PGconn *conn, char *action, int node, char *type, int upstream_node, char *cluster_name, char *node_name, char *conninfo, int priority, char *slot_name, bool active)
create_node_record(PGconn *conn, char *action, int node, char *type, int upstream_node, char *cluster_name, char *node_name, char *conninfo, int priority, char *slot_name)
{
char sqlquery[QUERY_STR_LEN];
char upstream_node_id[MAXLEN];
@@ -1342,9 +1241,8 @@ create_node_record(PGconn *conn, char *action, int node, char *type, int upstrea
sqlquery_snprintf(sqlquery,
"INSERT INTO %s.repl_nodes "
" (id, type, upstream_node_id, cluster, "
" name, conninfo, slot_name, "
" priority, active) "
"VALUES (%i, '%s', %s, '%s', '%s', '%s', %s, %i, %s) ",
" name, conninfo, slot_name, priority) "
"VALUES (%i, '%s', %s, '%s', '%s', '%s', %s, %i) ",
get_repmgr_schema_quoted(conn),
node,
type,
@@ -1353,8 +1251,7 @@ create_node_record(PGconn *conn, char *action, int node, char *type, int upstrea
node_name,
conninfo,
slot_name_buf,
priority,
active == true ? "TRUE" : "FALSE");
priority);
log_verbose(LOG_DEBUG, "create_node_record(): %s\n", sqlquery);
@@ -1394,7 +1291,7 @@ delete_node_record(PGconn *conn, int node, char *action)
if (action != NULL)
{
log_verbose(LOG_DEBUG, "delete_node_record(): action is \"%s\"\n", action);
log_verbose(LOG_DEBUG, "create_node_record(): action is \"%s\"\n", action);
}
res = PQexec(conn, sqlquery);
@@ -1489,6 +1386,7 @@ create_event_record(PGconn *conn, t_configuration_options *options, int node_id,
PQerrorMessage(conn));
success = false;
}
else
{
@@ -1714,12 +1612,12 @@ int
get_node_record(PGconn *conn, char *cluster, int node_id, t_node_info *node_info)
{
char sqlquery[QUERY_STR_LEN];
int result;
PGresult *res;
int ntuples;
sqlquery_snprintf(
sqlquery,
"SELECT id, type, upstream_node_id, name, conninfo, "
" slot_name, priority, active"
"SELECT id, type, upstream_node_id, name, conninfo, slot_name, priority, active"
" FROM %s.repl_nodes "
" WHERE cluster = '%s' "
" AND id = %i",
@@ -1729,51 +1627,6 @@ get_node_record(PGconn *conn, char *cluster, int node_id, t_node_info *node_info
log_verbose(LOG_DEBUG, "get_node_record():\n%s\n", sqlquery);
result = _get_node_record(conn, cluster, sqlquery, node_info);
if (result == 0)
{
log_verbose(LOG_DEBUG, "get_node_record(): no record found for node %i\n", node_id);
}
return result;
}
int
get_node_record_by_name(PGconn *conn, char *cluster, const char *node_name, t_node_info *node_info)
{
char sqlquery[QUERY_STR_LEN];
int result;
sqlquery_snprintf(
sqlquery,
"SELECT id, type, upstream_node_id, name, conninfo, slot_name, priority, active"
" FROM %s.repl_nodes "
" WHERE cluster = '%s' "
" AND name = '%s'",
get_repmgr_schema_quoted(conn),
cluster,
node_name);
log_verbose(LOG_DEBUG, "get_node_record_by_name():\n%s\n", sqlquery);
result = _get_node_record(conn, cluster, sqlquery, node_info);
if (result == 0)
{
log_verbose(LOG_DEBUG, "get_node_record(): no record found for node %s\n", node_name);
}
return result;
}
static int
_get_node_record(PGconn *conn, char *cluster, char *sqlquery, t_node_info *node_info)
{
int ntuples;
PGresult *res;
res = PQexec(conn, sqlquery);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
@@ -1784,6 +1637,7 @@ _get_node_record(PGconn *conn, char *cluster, char *sqlquery, t_node_info *node_
if (ntuples == 0)
{
log_verbose(LOG_DEBUG, "get_node_record(): no record found for node %i\n", node_id);
return 0;
}
@@ -1804,9 +1658,6 @@ _get_node_record(PGconn *conn, char *cluster, char *sqlquery, t_node_info *node_
}
int
get_node_replication_state(PGconn *conn, char *node_name, char *output)
{

View File

@@ -1,6 +1,5 @@
/*
* dbutils.h
*
* Copyright (c) 2ndQuadrant, 2010-2016
*
* This program is free software: you can redistribute it and/or modify
@@ -22,7 +21,6 @@
#define _REPMGR_DBUTILS_H_
#include "access/xlogdefs.h"
#include "pqexpbuffer.h"
#include "config.h"
#include "strutil.h"
@@ -54,6 +52,18 @@ typedef struct s_node_info
} t_node_info;
/*
* Struct to store replication slot information
*/
typedef struct s_replication_slot
{
char slot_name[MAXLEN];
char slot_type[MAXLEN];
bool active;
} t_replication_slot;
#define T_NODE_INFO_INITIALIZER { \
NODE_NOT_FOUND, \
NO_UPSTREAM_NODE, \
@@ -68,27 +78,13 @@ typedef struct s_node_info
InvalidXLogRecPtr \
}
/*
* Struct to store replication slot information
*/
typedef struct s_replication_slot
{
char slot_name[MAXLEN];
char slot_type[MAXLEN];
bool active;
} t_replication_slot;
extern char repmgr_schema[MAXLEN];
PGconn *_establish_db_connection(const char *conninfo,
const bool exit_on_error,
const bool log_notice,
const bool verbose_only);
const bool log_notice);
PGconn *establish_db_connection(const char *conninfo,
const bool exit_on_error);
PGconn *establish_db_connection_quiet(const char *conninfo);
PGconn *test_db_connection(const char *conninfo);
PGconn *test_db_connection(const char *conninfo,
const bool exit_on_error);
PGconn *establish_db_connection_by_params(const char *keywords[],
const char *values[],
const bool exit_on_error);
@@ -119,17 +115,16 @@ int wait_connection_availability(PGconn *conn, long long timeout);
bool cancel_query(PGconn *conn, int timeout);
char *get_repmgr_schema(void);
char *get_repmgr_schema_quoted(PGconn *conn);
bool create_replication_slot(PGconn *conn, char *slot_name, int server_version_num, PQExpBufferData *error_msg);
bool create_replication_slot(PGconn *conn, char *slot_name);
int get_slot_record(PGconn *conn, char *slot_name, t_replication_slot *record);
bool drop_replication_slot(PGconn *conn, char *slot_name);
bool start_backup(PGconn *conn, char *first_wal_segment, bool fast_checkpoint);
bool stop_backup(PGconn *conn, char *last_wal_segment);
bool set_config_bool(PGconn *conn, const char *config_param, bool state);
bool witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster_name);
bool create_node_record(PGconn *conn, char *action, int node, char *type, int upstream_node, char *cluster_name, char *node_name, char *conninfo, int priority, char *slot_name, bool active);
bool copy_configuration(PGconn *masterconn, PGconn *witnessconn, char *cluster_name);
bool create_node_record(PGconn *conn, char *action, int node, char *type, int upstream_node, char *cluster_name, char *node_name, char *conninfo, int priority, char *slot_name);
bool delete_node_record(PGconn *conn, int node, char *action);
int get_node_record(PGconn *conn, char *cluster, int node_id, t_node_info *node_info);
int get_node_record_by_name(PGconn *conn, char *cluster, const char *node_name, t_node_info *node_info);
bool update_node_record_status(PGconn *conn, char *cluster_name, int this_node_id, char *type, int upstream_node_id, bool active);
bool update_node_record_set_upstream(PGconn *conn, char *cluster_name, int this_node_id, int new_upstream_node_id);
bool create_event_record(PGconn *conn, t_configuration_options *options, int node_id, char *event, bool successful, char *details);
@@ -138,4 +133,3 @@ int get_node_replication_state(PGconn *conn, char *node_name, char *output)
t_server_type parse_node_type(const char *type);
int get_data_checksum_version(const char *data_directory);
#endif

View File

@@ -1,9 +1,9 @@
Package: repmgr-auto
Version: 3.2dev
Version: 3.0.1
Section: database
Priority: optional
Architecture: all
Depends: rsync, postgresql-9.3 | postgresql-9.4 | postgresql-9.5
Depends: rsync, postgresql-9.3 | postgresql-9.4
Maintainer: Self built package <user@localhost>
Description: PostgreSQL replication setup, magament and monitoring
has two main executables

194
dirmod.c
View File

@@ -1,194 +0,0 @@
/*
*
* dirmod.c
* directory handling functions
*
* Copyright (C) 2ndQuadrant, 2010-2016
*
* Portions Copyright (c) 1996-2013, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "postgres_fe.h"
/* Don't modify declarations in system headers */
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
/*
* pgfnames
*
* return a list of the names of objects in the argument directory. Caller
* must call pgfnames_cleanup later to free the memory allocated by this
* function.
*/
char **
pgfnames(const char *path)
{
DIR *dir;
struct dirent *file;
char **filenames;
int numnames = 0;
int fnsize = 200; /* enough for many small dbs */
dir = opendir(path);
if (dir == NULL)
{
return NULL;
}
filenames = (char **) palloc(fnsize * sizeof(char *));
while (errno = 0, (file = readdir(dir)) != NULL)
{
if (strcmp(file->d_name, ".") != 0 && strcmp(file->d_name, "..") != 0)
{
if (numnames + 1 >= fnsize)
{
fnsize *= 2;
filenames = (char **) repalloc(filenames,
fnsize * sizeof(char *));
}
filenames[numnames++] = pstrdup(file->d_name);
}
}
if (errno)
{
fprintf(stderr, _("could not read directory \"%s\": %s\n"),
path, strerror(errno));
}
filenames[numnames] = NULL;
if (closedir(dir))
{
fprintf(stderr, _("could not close directory \"%s\": %s\n"),
path, strerror(errno));
}
return filenames;
}
/*
* pgfnames_cleanup
*
* deallocate memory used for filenames
*/
void
pgfnames_cleanup(char **filenames)
{
char **fn;
for (fn = filenames; *fn; fn++)
pfree(*fn);
pfree(filenames);
}
/*
* rmtree
*
* Delete a directory tree recursively.
* Assumes path points to a valid directory.
* Deletes everything under path.
* If rmtopdir is true deletes the directory too.
* Returns true if successful, false if there was any problem.
* (The details of the problem are reported already, so caller
* doesn't really have to say anything more, but most do.)
*/
bool
rmtree(const char *path, bool rmtopdir)
{
bool result = true;
char pathbuf[MAXPGPATH];
char **filenames;
char **filename;
struct stat statbuf;
/*
* we copy all the names out of the directory before we start modifying
* it.
*/
filenames = pgfnames(path);
if (filenames == NULL)
return false;
/* now we have the names we can start removing things */
for (filename = filenames; *filename; filename++)
{
snprintf(pathbuf, MAXPGPATH, "%s/%s", path, *filename);
/*
* It's ok if the file is not there anymore; we were just about to
* delete it anyway.
*
* This is not an academic possibility. One scenario where this
* happens is when bgwriter has a pending unlink request for a file in
* a database that's being dropped. In dropdb(), we call
* ForgetDatabaseFsyncRequests() to flush out any such pending unlink
* requests, but because that's asynchronous, it's not guaranteed that
* the bgwriter receives the message in time.
*/
if (lstat(pathbuf, &statbuf) != 0)
{
if (errno != ENOENT)
{
result = false;
}
continue;
}
if (S_ISDIR(statbuf.st_mode))
{
/* call ourselves recursively for a directory */
if (!rmtree(pathbuf, true))
{
/* we already reported the error */
result = false;
}
}
else
{
if (unlink(pathbuf) != 0)
{
if (errno != ENOENT)
{
result = false;
}
}
}
}
if (rmtopdir)
{
if (rmdir(path) != 0)
{
result = false;
}
}
pgfnames_cleanup(filenames);
return result;
}

View File

@@ -1,23 +0,0 @@
/*
* dirmod.h
* Copyright (c) 2ndQuadrant, 2010-2016
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifndef _DIRMOD_H_
#define _DIRMOD_H_
#endif

View File

@@ -1,75 +0,0 @@
repmgrd's failover algorithm
============================
When implementing automatic failover, there are two factors which are critical in
ensuring the desired result is achieved:
- has the master node genuinely failed?
- which is the best node to promote to the new master?
This document outlines repmgrd's decision-making process during automatic failover
for standbys directly connected to the master node.
Master node failure detection
-----------------------------
If a `repmgrd` instance running on a PostgreSQL standby node is unable to connect to
the master node, this doesn't neccesarily mean that the master is down and a
failover is required. Factors such as network connectivity issues could mean that
even though the standby node is isolated, the replication cluster as a whole
is functioning correctly, and promoting the standby without further verification
could result in a "split-brain" situation.
In the event that `repmgrd` is unable to connect to the master node, it will attempt
to reconnect to the master server several times (as defined by the `reconnect_attempts`
parameter in `repmgr.conf`), with reconnection attempts occuring at the interval
specified by `reconnect_interval`. This happens to verify that the master is definitively
not accessible (e.g. that connection was not lost due to a brief network glitch).
Appropriate values for these settings will depend very much on the replication
cluster environment. There will necessarily be a trade-off between the time it
takes to assume the master is not reachable, and the reliability of that conclusion.
A standby in a different physical location to the master will probably need a longer
check interval to rule out possible network issues, whereas one located in the same
rack with a direct connection between servers could perform the check very quickly.
Note that it's possible the master comes back online after this point is reached,
but before a new master has been selected; in this case it will be noticed
during the selection of a new master and no actual failover will take place.
Promotion candidate selection
-----------------------------
Once `repmgrd` has decided the master is definitively unreachable, following checks
will be carried out:
* attempts to connect to all other nodes in the cluster (including the witness
node, if defined) to establish the state of the cluster, including their
current LSN
* If less than half of the nodes are visible (from the viewpoint
of this node), `repmgrd` will not take any further action. This is to ensure that
e.g. if a replication cluster is spread over multiple data centres, a split-brain
situation does not occur if there is a network failure between datacentres. Note
that if nodes are split evenly between data centres, a witness server can be
used to establish the "majority" daat centre.
* `repmgrd` polls all visible servers and waits for each node to return a valid LSN;
it updates the LSN previously stored for this node if it has increased since
the initial check
* once all LSNs have been retrieved, `repmgrd` will check for the highest LSN; if
its own node has the highest LSN, it will attempt to promote itself (using the
command defined in `promote_command` in `repmgr.conf`. Note that if using
`repmgr standby promote` as the promotion command, and the original master becomes available
before the promotion takes effect, `repmgr` will return an error and no promotion
will take place, and `repmgrd` will resume monitoring as usual.
* if the node is not the promotion candidate, `repmgrd` will execute the
`follow_command` defined in `repmgr.conf`. If using `repmgr standby follow` here,
`repmgr` will attempt to detect the new master node and attach to that.

View File

@@ -1,150 +0,0 @@
Fencing a failed master node with repmgrd and pgbouncer
=======================================================
With automatic failover, it's essential to ensure that a failed master
remains inaccessible to your application, even if it comes back online
again, to avoid a split-brain situation.
By using `pgbouncer` together with `repmgrd`, it's possible to combine
automatic failover with a process to isolate the failed master from
your application and ensure that all connections which should go to
the master are directed there smoothly without having to reconfigure
your application. (Note that as a connection pooler, `pgbouncer` can
benefit your application in other ways, but those are beyond the scope
of this document).
* * *
> *WARNING*: automatic failover is tricky to get right. This document
> demonstrates one possible implementation method, however you should
> carefully configure and test any setup to suit the needs of your own
> replication cluster/application.
* * *
In a failover situation, `repmgrd` promotes a standby to master by
executing the command defined in `promote_command`. Normally this
would be something like:
repmgr standby promote -f /etc/repmgr.conf
By wrapping this in a custom script which adjusts the `pgbouncer`
configuration on all nodes, it's possible to fence the failed master
and redirect write connections to the new master.
The script consists of three sections:
* commands to pause `pgbouncer` on all nodes
* the promotion command itself
* commands to reconfigure and restart `pgbouncer` on all nodes
Note that it requires password-less SSH access between all nodes to be
able to update the `pgbouncer` configuration files.
For the purposes of this demonstration, we'll assume there are 3 nodes
(master and two standbys), with `pgbouncer` listening on port 6432
handling connections to a database called `appdb`. The `postgres`
system user must have write access to the `pgbouncer` configuration
file on all nodes, assumed to be at `/etc/pgbouncer.ini`.
The script also requires a template file containing global `pgbouncer`
configuration, which should looks something like this (adjust
settings appropriately for your environment):
`/var/lib/postgres/repmgr/pgbouncer.ini.template`
[pgbouncer]
logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = *
listen_port = 6532
unix_socket_dir = /tmp
auth_type = trust
auth_file = /etc/pgbouncer.auth
admin_users = postgres
stats_users = postgres
pool_mode = transaction
max_client_conn = 100
default_pool_size = 20
min_pool_size = 5
reserve_pool_size = 5
reserve_pool_timeout = 3
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
The actual script is as follows; adjust the configurable items as appropriate:
`/var/lib/postgres/repmgr/promote.sh`
#!/usr/bin/env bash
set -u
set -e
# Configurable items
PGBOUNCER_HOSTS="node1 node2 node3"
REPMGR_DB="repmgr"
REPMGR_USER="repmgr"
REPMGR_SCHEMA="repmgr_test"
PGBOUNCER_CONFIG="/etc/pgbouncer.ini"
PGBOUNCER_INI_TEMPLATE="/var/lib/postgres/repmgr/pgbouncer.ini.template"
PGBOUNCER_DATABASE="appdb"
# 1. Pause running pgbouncer instances
for HOST in $PGBOUNCER_HOSTS
do
psql -t -c "pause" -h $HOST -p $PORT -U postgres pgbouncer
done
# 2. Promote this node from standby to master
repmgr standby promote -f /etc/repmgr.conf
# 3. Reconfigure pgbouncer instances
PGBOUNCER_INI_NEW="/tmp/pgbouncer.ini.new"
for HOST in $PGBOUNCER_HOSTS
do
# Recreate the pgbouncer config file
echo -e "[databases]\n" > $PGBOUNCER_INI_NEW
psql -d $REPMGR_DB -U $REPMGR_USER -t -A \
-c "SELECT '$PGBOUNCER_DATABASE= ' || conninfo || ' application_name=pgbouncer_$HOST' \
FROM $REPMGR_SCHEMA.repl_nodes \
WHERE active = TRUE AND type='master'" >> $PGBOUNCER_INI_NEW
cat $PGBOUNCER_INI_TEMPLATE >> $PGBOUNCER_INI_NEW
rsync $PGBOUNCER_INI_NEW $HOST:$PGBOUNCER_CONFIG
psql -tc "reload" -h $HOST -U postgres pgbouncer
psql -tc "resume" -h $HOST -U postgres pgbouncer
done
# Clean up generated file
rm $PGBOUNCER_INI_NEW
echo "Reconfiguration of pgbouncer complete"
Script and template file should be installed on each node where
`repmgrd` is running.
Finally, set `promote_command` in `repmgr.conf` on each node to
point to the custom promote script:
promote_command=/var/lib/postgres/repmgr/promote.sh
and reload/restart any running `repmgrd` instances for the changes to take
effect.

View File

@@ -29,6 +29,7 @@
#define ERR_DB_CON 6
#define ERR_DB_QUERY 7
#define ERR_PROMOTED 8
#define ERR_BAD_PASSWORD 9
#define ERR_STR_OVERFLOW 10
#define ERR_FAILOVER_FAIL 11
#define ERR_BAD_SSH 12
@@ -36,10 +37,5 @@
#define ERR_BAD_BASEBACKUP 14
#define ERR_INTERNAL 15
#define ERR_MONITORING_FAIL 16
#define ERR_BAD_BACKUP_LABEL 17
#define ERR_SWITCHOVER_FAIL 18
#define ERR_BARMAN 19
#define ERR_REGISTRATION_SYNC 20
#endif /* _ERRCODE_H_ */

7
log.c
View File

@@ -40,8 +40,7 @@
/* #define REPMGR_DEBUG */
static int detect_log_facility(const char *facility);
static void _stderr_log_with_level(const char *level_name, int level, const char *fmt, va_list ap)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 0)));
static void _stderr_log_with_level(const char *level_name, int level, const char *fmt, va_list ap);
int log_type = REPMGR_STDERR;
int log_level = LOG_NOTICE;
@@ -49,7 +48,7 @@ int last_log_level = LOG_NOTICE;
int verbose_logging = false;
int terse_logging = false;
extern void
void
stderr_log_with_level(const char *level_name, int level, const char *fmt, ...)
{
va_list arglist;
@@ -142,7 +141,7 @@ log_verbose(int level, const char *fmt, ...)
bool
logger_init(t_configuration_options *opts, const char *ident)
logger_init(t_configuration_options * opts, const char *ident)
{
char *level = opts->loglevel;
char *facility = opts->logfacility;

12
log.h
View File

@@ -25,7 +25,7 @@
#define REPMGR_SYSLOG 1
#define REPMGR_STDERR 2
extern void
void
stderr_log_with_level(const char *level_name, int level, const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
@@ -123,14 +123,10 @@ bool logger_shutdown(void);
void logger_set_verbose(void);
void logger_set_terse(void);
void log_hint(const char *fmt, ...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
void log_verbose(int level, const char *fmt, ...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void log_hint(const char *fmt, ...);
void log_verbose(int level, const char *fmt, ...);
extern int log_type;
extern int log_level;
extern int verbose_logging;
extern int terse_logging;
#endif /* _REPMGR_LOG_H_ */
#endif

4911
repmgr.c

File diff suppressed because it is too large Load Diff

View File

@@ -15,29 +15,21 @@
# schema (pattern: "repmgr_{cluster}"); while this name will be quoted
# to preserve case, we recommend using lower case and avoiding whitespace
# to facilitate easier querying of the repmgr views and tables.
#cluster=example_cluster
cluster=example_cluster
# Node ID and name
# (Note: we recommend to avoid naming nodes after their initial
# replication function, as this will cause confusion when e.g.
# replication funcion, as this will cause confusion when e.g.
# "standby2" is promoted to primary)
#node=2 # a unique integer
#node_name=node2 # an arbitrary (but unique) string; we recommend using
node=2 # a unique integer
node_name=node2 # an arbitrary (but unique) string; we recommend using
# the server's hostname or another identifier unambiguously
# associated with the server to avoid confusion
# Database connection information as a conninfo string
# This must be accessible to all servers in the cluster; for details see:
#
# https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
#
#conninfo='host=192.168.204.104 dbname=repmgr user=repmgr'
#
# If repmgrd is in use, consider explicitly setting `connect_timeout` in the
# conninfo string to determine the length of time which elapses before
# a network connection attempt is abandoned; for details see:
#
# https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNECT-CONNECT-TIMEOUT
# http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
conninfo='host=192.168.204.104 dbname=repmgr_db user=repmgr_usr'
# Optional configuration items
# ============================
@@ -45,16 +37,15 @@
# Replication settings
# ---------------------
# When using cascading replication, a standby can connect to another
# upstream standby node which is specified by setting 'upstream_node'.
# In that case, the upstream node must exist before the new standby
# can be registered. If 'upstream_node' is not set, then the standby
# will connect directly to the primary node.
#upstream_node=1
# when using cascading replication and a standby is to be connected to an
# upstream standby, specify that node's ID with 'upstream_node'. The node
# must exist before the new standby can be registered. If a standby is
# to connect directly to a primary node, this parameter is not required.
upstream_node=1
# use physical replication slots - PostgreSQL 9.4 and later only
# (default: 0)
#use_replication_slots=0
use_replication_slots=0
# NOTE: 'max_replication_slots' should be configured for at least the
# number of standbys which will connect to the primary.
@@ -64,15 +55,15 @@
# Log level: possible values are DEBUG, INFO, NOTICE, WARNING, ERR, ALERT, CRIT or EMERG
# (default: NOTICE)
#loglevel=NOTICE
loglevel=NOTICE
# Logging facility: possible values are STDERR or - for Syslog integration - one of LOCAL0, LOCAL1, ..., LOCAL7, USER
# (default: STDERR)
#logfacility=STDERR
logfacility=STDERR
# stderr can be redirected to an arbitrary file:
#
#logfile='/var/log/repmgr/repmgr.log'
logfile='/var/log/repmgr/repmgr.log'
# event notifications can be passed to an arbitrary external program
# together with the following parameters:
@@ -86,12 +77,12 @@
# the values provided for "%t" and "%d" will probably contain spaces,
# so should be quoted in the provided command configuration, e.g.:
#
#event_notification_command='/path/to/some/script %n %e %s "%t" "%d"'
event_notification_command='/path/to/some/script %n %e %s "%t" "%d"'
# By default, all notifications will be passed; the notification types
# can be filtered to explicitly named ones:
#
#event_notifications=master_register,standby_register,witness_create
event_notifications=master_register,standby_register,witness_create
# Environment/command settings
@@ -99,45 +90,17 @@
# path to PostgreSQL binary directory (location of pg_ctl, pg_basebackup etc.)
# (if not provided, defaults to system $PATH)
#pg_bindir=/usr/bin/
#
# Debian/Ubuntu users: you will probably need to set this to the directory
# where `pg_ctl` is located, e.g. /usr/lib/postgresql/9.5/bin/
# service control commands
#
# repmgr provides options to override the default pg_ctl commands
# used to stop, start, restart, reload and promote the PostgreSQL cluster
#
# NOTE: These commands must be runnable on remote nodes as well for switchover
# to function correctly.
#
# If you use sudo, the user repmgr runs as (usually 'postgres') must have
# passwordless sudo access to execute the command
#
# For example, to use systemd, you may use the following configuration:
#
# # this is required when running sudo over ssh without -t:
# Defaults:postgres !requiretty
# postgres ALL = NOPASSWD: /usr/bin/systemctl stop postgresql-9.5, \
# /usr/bin/systemctl start postgresql-9.5, \
# /usr/bin/systemctl restart postgresql-9.5
#
# service_start_command = systemctl start postgresql-9.5
# service_stop_command = systemctl stop postgresql-9.5
# service_restart_command = systemctl restart postgresql-9.5
# service_reload_command = pg_ctlcluster 9.5 main reload
# service_promote_command = pg_ctlcluster 9.5 main promote
pg_bindir=/usr/bin/
# external command options
#rsync_options=--archive --checksum --compress --progress --rsh="ssh -o \"StrictHostKeyChecking no\""
#ssh_options=-o "StrictHostKeyChecking no"
rsync_options=--archive --checksum --compress --progress --rsh="ssh -o \"StrictHostKeyChecking no\""
ssh_options=-o "StrictHostKeyChecking no"
# external command arguments. Values shown are examples.
#pg_ctl_options='-s'
#pg_basebackup_options='--xlog-method=s'
pg_ctl_options='-s'
pg_basebackup_options='--xlog-method=s'
# Standby clone settings
@@ -149,10 +112,6 @@
#
# tablespace_mapping=/path/to/original/tablespace=/path/to/new/tablespace
# You can specify a restore_command to be used in the recovery.conf that
# will be placed in the cloned standby
#
# restore_command = cp /path/to/archived/wals/%f %p
# Failover settings (repmgrd)
# ---------------------------
@@ -160,41 +119,30 @@
# These settings are only applied when repmgrd is running. Values shown
# are defaults.
# monitoring interval in seconds; default is 2
#monitor_interval_secs=2
# Number of seconds to wait for a response from the primary server before
# deciding it has failed.
#master_response_timeout=60
master_response_timeout=60
# Number of attempts at what interval (in seconds) to try and
# connect to a server to establish its status (e.g. master
# during failover)
#reconnect_attempts=6
#reconnect_interval=10
reconnect_attempts=6
reconnect_interval=10
# Autofailover options
#failover=manual # one of 'automatic', 'manual' (default: manual)
# defines the action to take in the event of upstream failure
#
# 'automatic': repmgrd will automatically attempt to promote the
# node or follow the new upstream node
# 'manual': repmgrd will take no action and the mode will require
# manual attention to reattach it to replication
#priority=100 # indicate a preferred priorty for promoting nodes
# a value of zero or less prevents the node being promoted to primary
failover=manual # one of 'automatic', 'manual'
# (default: manual)
priority=100 # a value of zero or less prevents the node being promoted to primary
# (default: 100)
promote_command='repmgr standby promote -f /path/to/repmgr.conf'
follow_command='repmgr standby follow -f /path/to/repmgr.conf -W'
#promote_command='repmgr standby promote -f /path/to/repmgr.conf'
#follow_command='repmgr standby follow -f /path/to/repmgr.conf -W'
# monitoring interval in seconds; default is 2
monitor_interval_secs=2
# change wait time for primary; before we bail out and exit when the primary
# disappears, we wait 'reconnect_attempts' * 'retry_promote_interval_secs'
# seconds; by default this would be half an hour, as 'retry_promote_interval_secs'
# default value is 300)
#retry_promote_interval_secs=300
# Number of seconds after which the witness server resyncs the repl_nodes table
#witness_repl_nodes_sync_interval_secs=15
retry_promote_interval_secs=300

121
repmgr.h
View File

@@ -23,20 +23,20 @@
#include <libpq-fe.h>
#include <postgres_fe.h>
#include <getopt_long.h>
#include "pqexpbuffer.h"
#include "strutil.h"
#include "dbutils.h"
#include "errcode.h"
#include "config.h"
#include "dirmod.h"
#define MIN_SUPPORTED_VERSION "9.3"
#define MIN_SUPPORTED_VERSION_NUM 90300
#include "config.h"
#define MAXFILENAME 1024
#define ERRBUFF_SIZE 512
#define DEFAULT_WAL_KEEP_SEGMENTS "0"
#define DEFAULT_WAL_KEEP_SEGMENTS "5000"
#define DEFAULT_DEST_DIR "."
#define DEFAULT_REPMGR_SCHEMA_PREFIX "repmgr_"
#define DEFAULT_PRIORITY 100
@@ -48,37 +48,17 @@
#define NO_UPSTREAM_NODE -1
#define UNKNOWN_NODE_ID -1
/* command line options without short versions */
#define OPT_HELP 1
#define OPT_CHECK_UPSTREAM_CONFIG 2
#define OPT_RECOVERY_MIN_APPLY_DELAY 3
#define OPT_COPY_EXTERNAL_CONFIG_FILES 4
#define OPT_CONFIG_ARCHIVE_DIR 5
#define OPT_PG_REWIND 6
#define OPT_PWPROMPT 7
#define OPT_CSV 8
#define OPT_NODE 9
#define OPT_WITHOUT_BARMAN 10
#define OPT_NO_UPSTREAM_CONNECTION 11
#define OPT_REGISTER_WAIT 12
#define OPT_CLUSTER 13
/* deprecated command line options */
#define OPT_INITDB_NO_PWPROMPT 999
#define OPT_IGNORE_EXTERNAL_CONFIG_FILES 998
#define CONFIG_FILE_SAMEPATH 1
#define CONFIG_FILE_PGDATA 2
/* Run time options type */
typedef struct
{
char dbname[MAXLEN];
char host[MAXLEN];
char username[MAXLEN];
char dest_dir[MAXPGPATH];
char config_file[MAXPGPATH];
char dest_dir[MAXFILENAME];
char config_file[MAXFILENAME];
char remote_user[MAXLEN];
char superuser[MAXLEN];
char wal_keep_segments[MAXLEN];
@@ -90,13 +70,8 @@ typedef struct
bool witness_pwprompt;
bool rsync_only;
bool fast_checkpoint;
bool csv_mode;
bool without_barman;
bool no_upstream_connection;
bool copy_external_config_files;
int copy_external_config_files_destination;
bool wait_register_sync;
int wait_register_sync_seconds;
bool ignore_external_config_files;
char pg_ctl_mode[MAXLEN];
char masterport[MAXLEN];
/*
* configuration file parameters which can be overridden on the
@@ -106,88 +81,24 @@ typedef struct
/* parameter used by STANDBY SWITCHOVER */
char remote_config_file[MAXLEN];
char pg_rewind[MAXPGPATH];
char pg_ctl_mode[MAXLEN];
char pg_rewind[MAXFILENAME];
/* parameter used by STANDBY {ARCHIVE_CONFIG | RESTORE_CONFIG} */
char config_archive_dir[MAXLEN];
/* parameter used by CLUSTER CLEANUP */
int keep_history;
/* parameter used by {STANDBY|WITNESS} UNREGISTER */
int node;
char pg_bindir[MAXLEN];
char recovery_min_apply_delay[MAXLEN];
/* deprecated command line options */
char localport[MAXLEN];
bool initdb_no_pwprompt;
} t_runtime_options;
#define T_RUNTIME_OPTIONS_INITIALIZER { "", "", "", "", "", "", "", DEFAULT_WAL_KEEP_SEGMENTS, false, false, false, false, false, false, false, false, false, false, false, false, CONFIG_FILE_SAMEPATH, false, 0, "", "", "", "", "fast", "", 0, UNKNOWN_NODE_ID, "", ""}
struct BackupLabel
{
XLogRecPtr start_wal_location;
char start_wal_file[MAXLEN];
XLogRecPtr checkpoint_location;
char backup_from[MAXLEN];
char backup_method[MAXLEN];
char start_time[MAXLEN];
char label[MAXLEN];
XLogRecPtr min_failover_slot_lsn;
};
typedef struct
{
char slot[MAXLEN];
char xlog_method[MAXLEN];
} t_basebackup_options;
#define T_BASEBACKUP_OPTIONS_INITIALIZER { "", "" }
typedef struct
{
int size;
char **keywords;
char **values;
} t_conninfo_param_list;
typedef struct
{
char filepath[MAXPGPATH];
char filename[MAXPGPATH];
bool in_data_directory;
} t_configfile_info;
typedef struct
{
int size;
int entries;
t_configfile_info **files;
} t_configfile_list;
#define T_CONFIGFILE_LIST_INITIALIZER { 0, 0, NULL }
typedef struct
{
int node_id;
int node_status;
} t_node_status_rec;
typedef struct
{
int node_id;
char node_name[MAXLEN];
t_node_status_rec **node_status_list;
} t_node_matrix_rec;
typedef struct
{
int node_id;
char node_name[MAXLEN];
t_node_matrix_rec **matrix_list_rec;
} t_node_status_cube;
#define T_RUNTIME_OPTIONS_INITIALIZER { "", "", "", "", "", "", "", DEFAULT_WAL_KEEP_SEGMENTS, false, false, false, false, false, false, false, false, false, "smart", "", "", "", "", "", 0, "", "", "", false }
extern char repmgr_schema[MAXLEN];
extern bool config_file_found;
#endif

View File

@@ -64,7 +64,7 @@ CREATE INDEX idx_repl_status_sort ON repl_monitor(last_monitor_time, standby_nod
* This view shows the list of nodes with the information of which one is the upstream
* in each case (when appliable)
*/
CREATE VIEW repl_show_nodes AS
CREATE VIEW repl_show_nodes AS
SELECT rn.id, rn.conninfo, rn.type, rn.name, rn.cluster,
rn.priority, rn.active, sq.name AS upstream_node_name
FROM repl_nodes as rn LEFT JOIN repl_nodes AS sq ON sq.id=rn.upstream_node_id;

706
repmgrd.c

File diff suppressed because it is too large Load Diff

View File

@@ -63,15 +63,6 @@ UPDATE repl_nodes SET type = 'master' WHERE id = $master_id;
-- UPDATE repl_nodes SET active = FALSE WHERE id IN (...);
/* There's also an event table which we need to create */
CREATE TABLE repl_events (
node_id INTEGER NOT NULL,
event TEXT NOT NULL,
successful BOOLEAN NOT NULL DEFAULT TRUE,
event_timestamp TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
details TEXT NULL
);
/* When you're sure of your changes, commit them */
-- COMMIT;

View File

@@ -1,32 +0,0 @@
/*
* Update a repmgr 3.1.1 installation to repmgr 3.1.2
* --------------------------------------------------
*
* This update is only required if repmgrd is being used in conjunction
* with a witness server.
*
* The new repmgr package should be installed first. Then
* carry out these steps:
*
* 1. (If repmgrd is used) stop any running repmgrd instances
* 2. On the master node, execute the SQL statement listed below
* 3. (If repmgrd is used) restart repmgrd
*/
/*
* If your repmgr installation is not included in your repmgr
* user's search path, please set the search path to the name
* of the repmgr schema to ensure objects are installed in
* the correct location.
*
* The repmgr schema is "repmgr_" + the cluster name defined in
* 'repmgr.conf'.
*/
-- SET search_path TO 'name_of_repmgr_schema';
BEGIN;
ALTER TABLE repl_nodes DROP CONSTRAINT repl_nodes_upstream_node_id_fkey,
ADD CONSTRAINT repl_nodes_upstream_node_id_fkey FOREIGN KEY (upstream_node_id) REFERENCES repl_nodes(id) DEFERRABLE;
COMMIT;

View File

@@ -83,12 +83,7 @@ _PG_init(void)
* resources in repmgr_shmem_startup().
*/
RequestAddinShmemSpace(repmgr_memsize());
#if (PG_VERSION_NUM >= 90600)
RequestNamedLWLockTranche("repmgr", 1);
#else
RequestAddinLWLocks(1);
#endif
/*
* Install hooks.
@@ -133,11 +128,7 @@ repmgr_shmem_startup(void)
if (!found)
{
/* First time through ... */
#if (PG_VERSION_NUM >= 90600)
shared_state->lock = &(GetNamedLWLockTranche("repmgr"))->lock;
#else
shared_state->lock = LWLockAssign();
#endif
snprintf(shared_state->location,
sizeof(shared_state->location), "%X/%X", 0, 0);
}

View File

@@ -87,34 +87,3 @@ maxlen_snprintf(char *str, const char *format,...)
return retval;
}
/*
* Adapted from: src/fe_utils/string_utils.c
*
* Function not publicly available before PostgreSQL 9.6.
*/
void
appendShellString(PQExpBuffer buf, const char *str)
{
const char *p;
appendPQExpBufferChar(buf, '\'');
for (p = str; *p; p++)
{
if (*p == '\n' || *p == '\r')
{
fprintf(stderr,
_("shell command argument contains a newline or carriage return: \"%s\"\n"),
str);
exit(ERR_BAD_CONFIG);
}
if (*p == '\'')
appendPQExpBufferStr(buf, "'\"'\"'");
else
appendPQExpBufferChar(buf, *p);
}
appendPQExpBufferChar(buf, '\'');
}

View File

@@ -22,20 +22,14 @@
#define _STRUTIL_H_
#include <stdlib.h>
#include "pqexpbuffer.h"
#include "errcode.h"
#define QUERY_STR_LEN 8192
#define MAXLEN 1024
#define MAXLINELENGTH 4096
#define MAXVERSIONSTR 16
#define MAXCONNINFO 1024
/* Why? http://stackoverflow.com/a/5459929/398670 */
#define STR(x) CppAsString(x)
#define MAXLEN_STR STR(MAXLEN)
extern int
xsnprintf(char *str, size_t size, const char *format,...)
@@ -49,6 +43,4 @@ extern int
maxlen_snprintf(char *str, const char *format,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
extern void
appendShellString(PQExpBuffer buf, const char *str);
#endif /* _STRUTIL_H_ */

View File

@@ -1,6 +1,6 @@
#ifndef _VERSION_H_
#define _VERSION_H_
#define REPMGR_VERSION "3.2"
#define REPMGR_VERSION "3.1.1"
#endif