README: clarify that the repmgr metadatabase must be part of the replication cluster

Bump version to 3.2.1

.gitignore

@@ -1,6 +1,9 @@
 *~
 *.o
+*.so
 repmgr
 repmgrd
 README.htm*
 README.pdf
+sql/repmgr_funcs.so
+sql/repmgr_funcs.sql

CONTRIBUTING.md

@@ -0,0 +1,29 @@
+License and Contributions
+=========================
+
+`repmgr` is licensed under the GPL v3.  All of its code and documentation is
+Copyright 2010-2016, 2ndQuadrant Limited.  See the files COPYRIGHT and LICENSE for
+details.
+
+The development of repmgr has primarily been sponsored by 2ndQuadrant customers.
+
+Additional work has been sponsored by the 4CaaST project for cloud computing,
+which has received funding from the European Union's Seventh Framework Programme
+(FP7/2007-2013) under grant agreement 258862.
+
+Contributions to `repmgr` are welcome, and will be listed in the file `CREDITS`.
+2ndQuadrant Limited requires that any contributions provide a copyright
+assignment and a disclaimer of any work-for-hire ownership claims from the
+employer of the developer.  This lets us make sure that all of the repmgr
+distribution remains free code.  Please contact info@2ndQuadrant.com for a
+copy of the relevant Copyright Assignment Form.
+
+Code style
+----------
+
+Code in repmgr is formatted to a consistent style using the following command:
+
+    astyle --style=ansi --indent=tab --suffix=none *.c *.h
+
+Contributors should reformat their code similarly before submitting code to
+the project, in order to minimize merge conflicts with other work.

COPYRIGHT

@@ -1,4 +1,4 @@
-Copyright (c) 2010-2011, 2ndQuadrant Limited
+Copyright (c) 2010-2016, 2ndQuadrant Limited
 All rights reserved.
 
 This program is free software: you can redistribute it and/or modify

CREDITS

@@ -10,3 +10,7 @@ Hannu Krosing <hannu@2ndQuadrant.com>
 Cédric Villemain <cedric@2ndquadrant.com>
 Charles Duffy <charles@dyfis.net>
 Daniel Farina <daniel@heroku.com>
+Shawn Ellis <shawn.ellis17@gmail.com>
+Jay Taylor <jay@jaytaylor.com>
+Christian Kruse <christian@2ndQuadrant.com>
+Krzysztof Gajdemski <songo@debian.org.pl>

FAILOVER.rst

@@ -0,0 +1 @@
+The contents of this file have been incorporated into the main README.md document.

FAQ.md

@@ -0,0 +1,183 @@
+FAQ - Frequently Asked Questions about repmgr
+=============================================
+
+This FAQ applies to `repmgr` 3.0 and later.
+
+General
+-------
+
+- What's the difference between the repmgr versions?
+
+  repmgr 3.x builds on the improved replication facilities added
+  in PostgreSQL 9.3, as well as improved automated failover support
+  via `repmgrd`, and is not compatible with PostgreSQL 9.2 and earlier.
+
+  repmgr 2.x supports PostgreSQL 9.0 onwards. While it is compatible
+  with  PostgreSQL 9.3 and later, we recommend repmgr v3.
+
+- What's the advantage of using replication slots?
+
+  Replication slots, introduced in PostgreSQL 9.4, ensure that the
+  master server will retain WAL files until they have been consumed
+  by all standby servers. This makes WAL file management much easier,
+  and if used `repmgr` will no longer insist on a fixed number (default: 5000)
+  of WAL files being preserved.
+
+  (However this does mean that if a standby is no longer connected to the
+  master, the master will retain WAL files indefinitely).
+
+- How many replication slots should I define in `max_replication_slots`?
+
+  Normally at least same number as the number of standbys which will connect
+  to the node. Note that changes to `max_replication_slots` require a server
+  restart to take effect, and as there is no particular penalty for unused
+  replication slots, setting a higher figure will make adding new nodes
+  easier.
+
+- Does `repmgr` support hash indexes?
+
+  No. Hash indexes and replication do not mix well and their use is
+  explicitly discouraged; see:
+    https://www.postgresql.org/docs/current/interactive/sql-createindex.html#AEN74175
+
+`repmgr`
+--------
+
+- When should I use the --rsync-only option?
+
+  By default, `repmgr` uses `pg_basebackup` to clone a standby from
+  a master. However, `pg_basebackup` copies the entire data directory, which
+  can take some time depending on installation size. If you have an
+  existing but "stale" standby, `repmgr` can use `rsync` instead,
+  which means only changed or added files need to be copied.
+
+- Can I register an existing master/standby?
+
+  Yes, this is no problem.
+
+- How can a failed master be re-added as a standby?
+
+  This is a two-stage process. First, the failed master's data directory
+  must be re-synced with the current master; secondly the failed master
+  needs to be re-registered as a standby. The section "Converting a failed
+  master to a standby" in the `README.md` file contains more detailed
+  information on this process.
+
+- Is there an easy way to check my master server is correctly configured
+  for use with `repmgr`?
+
+  Yes - execute `repmgr` with the `--check-upstream-config` option, and it
+  will let you know which items in `postgresql.conf` need to be modified.
+
+- Even though I specified custom `rsync` options, `repmgr` appends
+  the `--checksum` - why?
+
+  When syncing a stale data directory from an active server, it's
+  essential that `rsync` compares the content of files rather than
+  just timestamp and size, to ensure that all changed files are
+  copied and prevent corruption.
+
+- When cloning a standby, how can I prevent `repmgr` from copying
+  `postgresql.conf` and `pg_hba.conf` from the PostgreSQL configuration
+  directory in `/etc`?
+
+  Use the command line option `--ignore-external-config-files`
+
+- How can I prevent `repmgr` from copying local configuration files
+  in the data directory?
+
+  If you're updating an existing but stale data directory which
+  contains e.g. configuration files you don't want to be overwritten
+  with the same file from the master, specify the files in the
+  `rsync_options` configuration option, e.g.
+
+      rsync_options=--exclude=postgresql.local.conf
+
+  This option is only available when using the `--rsync-only` option.
+
+- How can I make the witness server use a particular port?
+
+  By default the witness server is configured to use port 5499; this
+  is intended to support running the witness server as  a separate
+  instance on a normal node server, rather than on its own dedicated server.
+
+  To specify different port for the witness server, supply the port number
+  in the `conninfo` string in `repmgr.conf`
+  (repmgr 3.0.1 and earlier: use the `-l/--local-port` option)
+
+- Do I need to include `shared_preload_libraries = 'repmgr_funcs'`
+  in `postgresql.conf` if I'm not using `repmgrd`?
+
+  No, the `repmgr_funcs` library is only needed when running `repmgrd`.
+  If you later decide to run `repmgrd`, you just need to add
+  `shared_preload_libraries = 'repmgr_funcs'` and restart PostgreSQL.
+
+- I've provided replication permission for the `repmgr` user in `pg_hba.conf`
+  but `repmgr`/`repmgrd` complains it can't connect to the server... Why?
+
+  `repmgr`/`repmgrd` need to be able to connect to the repmgr database
+  with a normal connection to query metadata. The `replication` connection
+  permission is for PostgreSQL's streaming replication and doesn't
+  necessarily need to be the `repmgr` user.
+
+- When cloning a standby, why do I need to provide the connection parameters
+  for the primary server on the command line, not in the configuration file?
+
+  Cloning a standby is a one-time action; the role of the server being cloned
+  from could change, so fixing it in the configuration file would create
+  confusion. If `repmgr` needs to establish a connection to the primary
+  server, it can retrieve this from the `repl_nodes` table or if necessary
+  scan the replication cluster until it locates the active primary.
+
+- Why is there no foreign key on the `node_id` column in the `repl_events`
+  table?
+
+  Under some circumstances event notifications can be generated for servers
+  which have not yet been registered; it's also useful to retain a record
+  of events which includes servers removed from the replication cluster
+  which no longer have an entry in the `repl_nodes` table.
+
+
+`repmgrd`
+---------
+
+- Do I need a witness server?
+
+  Not necessarily. However if you have an uneven number of nodes spread
+  over more than one network segment, a witness server will enable
+  better handling of a 'split brain' situation by providing a "casting
+  vote" on the preferred network segment.
+
+- How can I prevent a node from ever being promoted to master?
+
+  In `repmgr.conf`, set its priority to a value of 0 or less.
+
+  Additionally, if `failover` is set to `manual`, the node will never
+  be considered as a promotion candidate.
+
+- Does `repmgrd` support delayed standbys?
+
+  `repmgrd` can monitor delayed standbys - those set up with
+  `recovery_min_apply_delay` set to a non-zero value in `recovery.conf` -
+  but as it's not currently possible to directly examine the value
+  applied to the standby, `repmgrd` may not be able to properly evaluate
+  the node as a promotion candidate.
+
+  We recommend that delayed standbys are explicitly excluded from promotion
+  by setting `priority` to 0 in `repmgr.conf`.
+
+  Note that after registering a delayed standby, `repmgrd` will only start
+  once the metadata added in the master node has been replicated.
+
+- How can I get `repmgrd` to rotate its logfile?
+
+  Configure your system's `logrotate` service to do this; see example
+  in README.md
+
+- I've recloned a failed master as a standby, but `repmgrd` refuses to start?
+
+  Check you registered the standby after recloning. If unregistered the standby
+  cannot be considered as a promotion candidate even if `failover` is set to
+  `automatic`, which is probably not what you want. `repmgrd` will start if
+  `failover` is set to `manual` so the node's replication status can still
+  be monitored, if desired.

HISTORY

@@ -1,7 +1,214 @@
-1.0.0   2010-12-05
-        First public release
+3.2.1   2016-10-24
+        repmgr: require a valid repmgr cluster name unless -F/--force
+          supplied (Ian)
+        repmgr: check master server is registered with repmgr before
+          cloning (Ian)
+        repmgr: ensure data directory defaults to that of the source node (Ian)
+        repmgr: various fixes to Barman cloning mode (Gianni, Ian)
+        repmgr: fix `repmgr cluster crosscheck` output (Ian)
 
-1.1.0b1 2011-02-24 
+3.2     2016-10-05
+        repmgr: add support for cloning from a Barman backup (Gianni)
+        repmgr: add commands `standby matrix` and `standby crosscheck` (Gianni)
+        repmgr: suppress connection error display in `repmgr cluster show`
+          unless `--verbose` supplied (Ian)
+        repmgr: add commands `witness register` and `witness unregister` (Ian)
+        repmgr: enable `standby unregister` / `witness unregister` to be
+          executed for a node which is not running (Ian)
+        repmgr: remove deprecated command line options --initdb-no-pwprompt and
+           -l/--local-port (Ian)
+        repmgr: before cloning with pg_basebackup, check that sufficient free
+           walsenders are available (Ian)
+        repmgr: add option `--wait-sync` for `standby register` which causes
+           repmgr to wait for the registered node record to synchronise to
+           the standby (Ian)
+        repmgr: add option `--copy-external-config-files` for files outside
+           of the data directory (Ian)
+        repmgr: only require `wal_keep_segments` to be set in certain corner
+           cases (Ian)
+        repmgr: better support cloning from a node other than the one to
+           stream from (Ian)
+        repmgrd: add configuration options to override the default pg_ctl
+           commands (Jarkko Oranen, Ian)
+        repmgrd: don't start if node is inactive and failover=automatic (Ian)
+        packaging: improve "repmgr-auto" Debian package (Gianni)
+
+
+3.1.5   2016-08-15
+        repmgrd: in a failover situation, prevent endless looping when
+          attempting to establish the status of a node with
+          `failover=manual` (Ian)
+        repmgrd: improve handling of failover events on standbys with
+          `failover=manual`, and create a new event notification
+          for this, `standby_disconnect_manual` (Ian)
+        repmgr: add further event notifications (Gianni)
+        repmgr: when executing `standby switchover`, don't collect remote
+          command output unless required (Gianni, Ian)
+        repmgrd: improve standby monitoring query (Ian, based on suggestion
+          from  Álvaro)
+        repmgr: various command line handling improvements (Ian)
+
+3.1.4   2016-07-12
+        repmgr: new configuration option for setting "restore_command"
+          in the recovery.conf file generated by repmgr (Martín)
+        repmgr: add --csv option to "repmgr cluster show" (Gianni)
+        repmgr: enable provision of a conninfo string as the -d/--dbname
+          parameter, similar to other PostgreSQL utilities (Ian)
+        repmgr: during switchover operations improve detection of
+          demotion candidate shutdown (Ian)
+        various bugfixes and documentation updates (Ian, Martín)
+
+3.1.3   2016-05-17
+        repmgrd: enable monitoring when a standby is catching up by
+          replaying archived WAL (Ian)
+        repmgrd: when upstream_node_id is NULL, assume upstream node
+          to be current master (Ian)
+        repmgrd: check for reappearance of the master node if standby
+          promotion fails (Ian)
+        improve handling of rsync failure conditions (Martín)
+
+3.1.2   2016-04-12
+        Fix pg_ctl path generation in do_standby_switchover() (Ian)
+        Regularly sync witness server repl_nodes table (Ian)
+        Documentation improvements (Gianni, dhyannataraj)
+        (Experimental) ensure repmgr handles failover slots when copying
+          in rsync mode (Craig, Ian)
+        rsync mode handling fixes (Martín)
+        Enable repmgr to compile against 9.6devel (Ian)
+
+3.1.1   2016-02-24
+        Add '-P/--pwprompt' option for "repmgr create witness" (Ian)
+        Prevent repmgr/repmgrd running as root (Ian)
+
+3.1.0   2016-02-01
+        Add "repmgr standby switchover" command (Ian)
+        Revised README file (Ian)
+        Remove requirement for 'archive_mode' to be enabled (Ian)
+        Improve -?/--help output, showing default values if relevant (Ian)
+        Various bugfixes to command line/configuration parameter handling (Ian)
+
+3.0.3   2016-01-04
+        Create replication slot if required before base backup is run (Abhijit)
+        standy clone: when using rsync, clean up "pg_replslot" directory (Ian)
+        Improve --help output (Ian)
+        Improve config file parsing (Ian)
+        Various logging output improvements, including explicit HINTS (Ian)
+        Add --log-level to explicitly set log level on command line (Ian)
+        Repurpose --verbose to display extra log output (Ian)
+        Add --terse to hide hints and other non-critical output (Ian)
+        Reference internal functions with explicit catalog path (Ian)
+        When following a new primary, have repmgr (not repmgrd) create the new slot (Ian)
+        Add /etc/repmgr.conf as a default configuration file location (Ian)
+        Prevent repmgrd's -v/--verbose option expecting a parameter (Ian)
+        Prevent invalid replication_lag values being written to the monitoring table (Ian)
+        Improve repmgrd behaviour when monitored standby node is temporarily
+          unavailable (Martín)
+
+3.0.2   2015-10-02
+        Improve handling of --help/--version options; and improve help output (Ian)
+        Improve handling of situation where logfile can't be opened (Ian)
+        Always pass -D/--pgdata option to pg_basebackup (Ian)
+        Bugfix: standby clone --force does not empty pg_xlog (Gianni)
+        Bugfix: autofailover with reconnect_attempts > 1 (Gianni)
+        Bugfix: ignore comments after values (soxwellfb)
+        Bugfix: handle string values in 'node' parameter correctly (Gregory Duchatelet)
+        Allow repmgr to be compiled with a newer libpq (Marco)
+        Bugfix: call update_node_record_set_upstream() for STANDBY FOLLOW (Tomas)
+        Update `repmgr --help` output (per Github report from renard)
+        Update tablespace remapping in --rsync-only mode for 9.5 and later (Ian)
+        Deprecate `-l/--local-port` option - the port can be extracted
+          from the conninfo string in repmgr.conf (Ian)
+        Add STANDBY UNREGISTER (Vik Fearing)
+        Don't fail with error when registering master if schema already defined (Ian)
+        Fixes to whitespace handling when parsing config file (Ian)
+
+3.0.1   2015-04-16
+        Prevent repmgrd from looping infinitely if node was not registered (Ian)
+        When promoting a standby, have repmgr (not repmgrd) handle metadata updates (Ian)
+        Re-use replication slot if it already exists (Ian)
+        Prevent a test SSH connection being made when not needed (Ian)
+        Correct monitoring table column names (Ian)
+
+3.0     2015-03-27
+        Require PostgreSQL 9.3 or later (Ian)
+        Use `pg_basebackup` by default (instead of `rsync`) to clone standby servers (Ian)
+        Use `pg_ctl promote` to promote a standby to primary
+        Enable tablespace remapping using `pg_basebackup` (in PostgreSQL 9.3 with `rsync`) (Ian)
+        Support cascaded standbys (Ian)
+        "pg_bindir" no longer required as a configuration parameter (Ian)
+        Enable replication slots to be used (PostgreSQL 9.4 and later (Ian)
+        Command line option "--check-upstream-config" (Ian)
+        Add event logging table and option to execute an external program when an event occurs (Ian)
+        General usability and logging message improvements (Ian)
+        Code consolidation and cleanup (Ian)
+
+2.0.3   2015-04-16
+        Add -S/--superuser option for witness database creation Ian)
+        Add -c/--fast-checkpoint option for cloning (Christoph)
+        Add option "--initdb-no-pwprompt" (Ian)
+
+2.0.2   2015-02-17
+        Add "--checksum" in rsync when using "--force" (Jaime)
+        Use createdb/createuser instead of psql (Jaime)
+        Fixes to witness creation and monitoring (wamonite)
+        Use default master port if none supplied (Martín)
+        Documentation fixes and improvements (Ian)
+
+2.0.1   2014-07-16
+        Documentation fixes and new QUICKSTART file (Ian)
+        Explicitly specify directories to ignore when cloning (Ian)
+        Fix log level for some log messages (Ian)
+        RHEL/CentOS specfile, init script and Makefile fixes (Nathan Van Overloop)
+        Debian init script and config file documentation fixes (József Kószó)
+        Typo fixes (Riegie Godwin Jeyaranchen, PriceChild)
+
+2.0stable 2014-01-30
+        Documentation fixes (Christian)
+        General refactoring, code quality improvements and stabilization work (Christian)
+        Added proper daemonizing (-d/--daemonize) (Christian)
+        Added PID file handling (-p/--pid-file) (Christian)
+        New config option: monitor_interval_secs (Christian)
+        New config option: retry_promote_interval (Christian)
+        New config option: logfile (Christian)
+        New config option: pg_bindir (Christian)
+        New config option: pgctl_options (Christian)
+
+2.0beta2 2013-12-19
+        Improve autofailover logic and algorithms (Jaime, Andres)
+        Ignore pg_log when cloning (Jaime)
+        Add timestamps to log line in stderr (Christian)
+        Correctly check wal_keep_segments (Jay Taylor)
+        Add a ssh_options parameter (Jay Taylor)
+
+2.0beta1 2012-07-27
+        Make CLONE command try to make an exact copy including $PGDATA location (Cedric) 
+        Add detection of master failure (Jaime)
+        Add the notion of a witness server (Jaime)
+        Add autofailover capabilities (Jaime)
+        Add a configuration parameter to indicate the script to execute on failover or follow (Jaime)
+        Make the monitoring optional and turned off by default, it can be turned on with --monitoring-history switch (Jaime)
+        Add tunables to specify number of retries to reconnect to master and the time between them (Jaime)
+
+1.2.0   2012-07-27
+        Test ssh connection before trying to rsync (Cédric)
+        Add CLUSTER SHOW command (Carlo)
+        Add CLUSTER CLEANUP command (Jaime)
+        Add function write_primary_conninfo (Marco)
+        Teach repmgr how to get tablespace's location in different pg version (Jaime)
+        Improve version message (Carlo)
+
+1.1.1   2012-04-18
+        Add --ignore-rsync-warning (Cédric)
+        Add strnlen for compatibility with OS X (Greg)
+        Improve performance of the repl_status view (Jaime)
+        Remove last argument from log_err (Jaime, Reported by Jeroen Dekkers)
+        Complete documentation about possible error conditions (Jaime)
+        Document how to clean history (Jaime)
+
+1.1.0   2011-03-09
+        Make options -U, -R and -p not mandatory (Jaime)
+
+1.1.0b1 2011-02-24
         Fix missing "--force" option in help (Greg Smith)
         Correct warning message for wal_keep_segments (Bas van Oostveen)
         Add Debian build/usage docs (Bas, Hannu Krosing, Cedric Villemain)
@@ -28,6 +235,5 @@
         Map old verbose flag into a useful setting for the new logger (Greg)
         Document repmgrd startup restrictions and log info about them (Greg)
 
-1.1.0   2011-03-09
-        Make options -U, -R and -p not mandatory (Jaime)
-
+1.0.0   2010-12-05
+        First public release

Makefile

@@ -1,25 +1,37 @@
 #
 # Makefile
-# Copyright (c) 2ndQuadrant, 2010-2011
+# Copyright (c) 2ndQuadrant, 2010-2016
+
+HEADERS = $(wildcard *.h)
 
 repmgrd_OBJS = dbutils.o config.o repmgrd.o log.o strutil.o
-repmgr_OBJS = dbutils.o check_dir.o config.o repmgr.o log.o strutil.o
+repmgr_OBJS = dbutils.o check_dir.o config.o repmgr.o log.o strutil.o dirmod.o
 
 DATA = repmgr.sql uninstall_repmgr.sql
 
 PG_CPPFLAGS = -I$(libpq_srcdir)
-PG_LIBS = $(libpq_pgport)
+PG_LIBS     = $(libpq_pgport)
 
-all:  repmgrd repmgr
+
+all: repmgrd repmgr
+	$(MAKE) -C sql
 
 repmgrd: $(repmgrd_OBJS)
-	$(CC) $(CFLAGS) $(repmgrd_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o repmgrd
+	$(CC) -o repmgrd $(CFLAGS) $(repmgrd_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS)
+	$(MAKE) -C sql
 
 repmgr: $(repmgr_OBJS)
-	$(CC) $(CFLAGS) $(repmgr_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o repmgr
+	$(CC) -o repmgr $(CFLAGS) $(repmgr_OBJS) $(PG_LIBS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS)
+
+# Make all objects depend on all include files. This is a bit of a
+# shotgun approach, but the codebase is small enough that a complete rebuild
+# is very fast anyway.
+$(repmgr_OBJS): $(HEADERS)
+$(repmgrd_OBJS): $(HEADERS)
 
 ifdef USE_PGXS
-PGXS := $(shell pg_config --pgxs)
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
 include $(PGXS)
 else
 subdir = contrib/repmgr
@@ -28,11 +40,34 @@ include $(top_builddir)/src/Makefile.global
 include $(top_srcdir)/contrib/contrib-global.mk
 endif
 
-# XXX: Try to use PROGRAM construct (see pgxs.mk) someday. Right now
-# is overriding pgxs install.
-install:
-	$(INSTALL_PROGRAM) repmgrd$(X) '$(DESTDIR)$(bindir)'
-	$(INSTALL_PROGRAM) repmgr$(X) '$(DESTDIR)$(bindir)'
+# XXX: This overrides the pgxs install target - we're building two binaries,
+# which is not supported by pgxs.mk's PROGRAM construct.
+install: install_prog install_ext
+
+install_prog:
+	mkdir -p '$(DESTDIR)$(bindir)'
+	$(INSTALL_PROGRAM) repmgrd$(X) '$(DESTDIR)$(bindir)/'
+	$(INSTALL_PROGRAM) repmgr$(X) '$(DESTDIR)$(bindir)/'
+
+install_ext:
+	$(MAKE) -C sql install
+
+# Distribution-specific package building targets
+# ----------------------------------------------
+#
+# XXX we recommend using the PGDG-supplied packages where possible;
+# see README.md for details.
+
+install_rhel:
+	mkdir -p '$(DESTDIR)/etc/init.d/'
+	$(INSTALL_PROGRAM) RHEL/repmgrd.init '$(DESTDIR)/etc/init.d/repmgrd'
+	mkdir -p '$(DESTDIR)/etc/sysconfig/'
+	$(INSTALL_PROGRAM) RHEL/repmgrd.sysconfig '$(DESTDIR)/etc/sysconfig/repmgrd'
+	mkdir -p '$(DESTDIR)/etc/repmgr/'
+	$(INSTALL_PROGRAM) repmgr.conf.sample '$(DESTDIR)/etc/repmgr/'
+	mkdir -p '$(DESTDIR)/usr/bin/'
+	$(INSTALL_PROGRAM) repmgrd$(X) '$(DESTDIR)/usr/bin/'
+	$(INSTALL_PROGRAM) repmgr$(X) '$(DESTDIR)/usr/bin/'
 
 ifneq (,$(DATA)$(DATA_built))
 	@for file in $(addprefix $(srcdir)/, $(DATA)) $(DATA_built); do \
@@ -45,10 +80,25 @@ clean:
 	rm -f *.o
 	rm -f repmgrd
 	rm -f repmgr
+	$(MAKE) -C sql clean
+
+# Get correct version numbers and install paths, depending on your postgres version
+PG_VERSION = $(shell pg_config --version | cut -d ' ' -f 2 | cut -d '.' -f 1,2)
+REPMGR_VERSION = $(shell grep REPMGR_VERSION version.h | cut -d ' ' -f 3 | cut -d '"' -f 2)
+PKGLIBDIR = $(shell pg_config --pkglibdir)
+SHAREDIR = $(shell pg_config --sharedir)
+PGBINDIR = /usr/lib/postgresql/$(PG_VERSION)/bin
 
 deb: repmgrd repmgr
-	mkdir -p ./debian/usr/bin
-	cp repmgrd repmgr ./debian/usr/bin/
+	mkdir -p ./debian/usr/bin ./debian$(PGBINDIR)
+	cp repmgrd repmgr ./debian$(PGBINDIR)
+	ln -s ../..$(PGBINDIR)/repmgr ./debian/usr/bin/repmgr
+	mkdir -p ./debian$(SHAREDIR)/contrib/
+	cp sql/repmgr_funcs.sql ./debian$(SHAREDIR)/contrib/
+	cp sql/uninstall_repmgr_funcs.sql ./debian$(SHAREDIR)/contrib/
+	mkdir -p ./debian$(PKGLIBDIR)/
+	cp sql/repmgr_funcs.so ./debian$(PKGLIBDIR)/
 	dpkg-deb --build debian
-	mv debian.deb ../postgresql-repmgr-9.0_1.0.0.deb
+	mv debian.deb ../postgresql-repmgr-$(PG_VERSION)_$(REPMGR_VERSION).deb
+	rm -rf ./debian/usr
 

PACKAGES.md

@@ -0,0 +1,127 @@
+Packaging
+=========
+
+Notes on RedHat Linux, Fedora, and CentOS Builds
+------------------------------------------------
+
+The RPM packages of PostgreSQL put `pg_config` into the `postgresql-devel`
+package, not the main server one.  And if you have a RPM install of PostgreSQL
+9.0, the entire PostgreSQL binary directory will not be in your PATH by default
+either.  Individual utilities are made available via the `alternatives`
+mechanism, but not all commands will be wrapped that way.  The files installed
+by repmgr will certainly not be in the default PATH for the postgres user
+on such a system.  They will instead be in /usr/pgsql-9.0/bin/ on this
+type of system.
+
+When building repmgr against a RPM packaged build, you may discover that some
+development packages are needed as well.  The following build errors can
+occur:
+
+    /usr/bin/ld: cannot find -lxslt
+    /usr/bin/ld: cannot find -lpam
+
+Install the following packages to correct those:
+
+
+    yum install libxslt-devel
+    yum install pam-devel
+
+If building repmgr as a regular user, then doing the install into the system
+directories using sudo, the syntax is hard.  `pg_config` won't be in root's
+path either.  The following recipe should work:
+
+    sudo PATH="/usr/pgsql-9.0/bin:$PATH" make USE_PGXS=1 install
+
+
+Issues with 32 and 64 bit RPMs
+------------------------------
+
+If when building, you receive a series of errors of this form:
+
+  /usr/bin/ld: skipping incompatible /usr/pgsql-9.0/lib/libpq.so when searching for -lpq
+
+This is likely because you have both the 32 and 64 bit versions of the
+`postgresql90-devel` package installed.  You can check that like this:
+
+    rpm -qa --queryformat '%{NAME}\t%{ARCH}\n'  | grep postgresql90-devel
+
+And if two packages appear, one for i386 and one for x86_64, that's not supposed
+to be allowed.
+
+This can happen when using the PGDG repo to install that package;
+here is an example sessions demonstrating the problem case appearing:
+
+
+    # yum install postgresql-devel
+    ..
+    Setting up Install Process
+    Resolving Dependencies
+    --> Running transaction check
+    ---> Package postgresql90-devel.i386 0:9.0.2-2PGDG.rhel5 set to be updated
+    ---> Package postgresql90-devel.x86_64 0:9.0.2-2PGDG.rhel5 set to be updated
+    --> Finished Dependency Resolution
+
+    Dependencies Resolved
+
+    =========================================================================
+     Package               Arch      Version              Repository    Size
+    =========================================================================
+    Installing:
+     postgresql90-devel    i386      9.0.2-2PGDG.rhel5    pgdg90        1.5 M
+     postgresql90-devel    x86_64    9.0.2-2PGDG.rhel5    pgdg90        1.6 M
+
+
+Note how both the i386 and x86_64 platform architectures are selected for
+installation.  Your main PostgreSQL package will only be compatible with one of
+those, and if the repmgr build finds the wrong postgresql90-devel these
+"skipping incompatible" messages appear.
+
+In this case, you can temporarily remove both packages, then just install the
+correct one for your architecture.  Example:
+
+    rpm -e postgresql90-devel --allmatches
+    yum install postgresql90-devel-9.0.2-2PGDG.rhel5.x86_64
+
+Instead just deleting the package from the wrong platform might not leave behind
+the correct files, due to the way in which these accidentally happen to interact.
+If you already tried to build repmgr before doing this, you'll need to do:
+
+    make USE_PGXS=1 clean
+
+to get rid of leftover files from the wrong architecture.
+
+Notes on Ubuntu, Debian or other Debian-based Builds
+----------------------------------------------------
+
+The Debian packages of PostgreSQL put `pg_config` into the development package
+called `postgresql-server-dev-$version`.
+
+When building repmgr against a Debian packages build, you may discover that some
+development packages are needed as well. You will need the following development
+packages installed:
+
+    sudo apt-get install libxslt-dev libxml2-dev libpam-dev libedit-dev
+
+If you're using Debian packages for PostgreSQL and are building repmgr with the
+USE_PGXS option you also need to install the corresponding development package:
+
+    sudo apt-get install postgresql-server-dev-9.0
+
+If you build and install repmgr manually it will not be on the system path. The
+binaries will be installed in /usr/lib/postgresql/$version/bin/ which is not on
+the default path. The reason behind this is that Ubuntu/Debian systems manage
+multiple installed versions of PostgreSQL on the same system through a wrapper
+called pg_wrapper and repmgr is not (yet) known to this wrapper.
+
+You can solve this in many different ways, the most Debian like is to make an
+alternate for repmgr and repmgrd:
+
+    sudo update-alternatives --install /usr/bin/repmgr repmgr /usr/lib/postgresql/9.0/bin/repmgr 10
+    sudo update-alternatives --install /usr/bin/repmgrd repmgrd /usr/lib/postgresql/9.0/bin/repmgrd 10
+
+You can also make a deb package of repmgr using:
+
+    make USE_PGXS=1 deb
+
+This will build a Debian package one level up from where you build, normally the
+same directory that you have your repmgr/ directory in.

QUICKSTART.md

@@ -0,0 +1 @@
+The contents of this file have been incorporated into the main README.md document.

SSH-RSYNC.md

@@ -0,0 +1,36 @@
+Set up trusted copy between postgres accounts
+---------------------------------------------
+
+If you need to use `rsync` to clone standby servers, the `postgres` account
+on your primary and standby servers must be each able to access the other
+using SSH without a password.
+
+First generate an ssh key, using an empty passphrase, and copy the resulting
+keys and a matching authorization file to a privileged user account on the other
+system:
+
+    [postgres@node1]$ ssh-keygen -t rsa
+    Generating public/private rsa key pair.
+    Enter file in which to save the key (/var/lib/pgsql/.ssh/id_rsa):
+    Enter passphrase (empty for no passphrase):
+    Enter same passphrase again:
+    Your identification has been saved in /var/lib/pgsql/.ssh/id_rsa.
+    Your public key has been saved in /var/lib/pgsql/.ssh/id_rsa.pub.
+    The key fingerprint is:
+    aa:bb:cc:dd:ee:ff:aa:11:22:33:44:55:66:77:88:99 postgres@db1.domain.com
+    [postgres@node1]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
+    [postgres@node1]$ chmod go-rwx ~/.ssh/*
+    [postgres@node1]$ cd ~/.ssh
+    [postgres@node1]$ scp id_rsa.pub id_rsa authorized_keys user@node2:
+
+Login as a user on the other system, and install the files into the `postgres`
+user's account:
+
+    [user@node2 ~]$ sudo chown postgres.postgres authorized_keys id_rsa.pub id_rsa
+    [user@node2 ~]$ sudo mkdir -p ~postgres/.ssh
+    [user@node2 ~]$ sudo chown postgres.postgres ~postgres/.ssh
+    [user@node2 ~]$ sudo mv authorized_keys id_rsa.pub id_rsa ~postgres/.ssh
+    [user@node2 ~]$ sudo chmod -R go-rwx ~postgres/.ssh
+
+Now test that ssh in both directions works.  You may have to accept some new
+known hosts in the process.

TODO

@@ -1,21 +1,74 @@
 Known issues in repmgr
 ======================
 
-* The check for whether ``wal_keep_segments`` is considered large enough
-  does a string comparison rather than an integer one.  It can give both
-  false positive (setting is large enough but flagged as too small) and
-  false negative (setting is too small but not noted as such) errors.
-
 * When running repmgr against a remote machine, operations that start
   the database server using the ``pg_ctl`` command may accidentally
   terminate after their associated ssh session ends.
 
-* After running repmgrd as a regular foreground application, hitting
-  control-C causes the program to crash.
+* PGPASSFILE may not be passed to pg_basebackup
+
 
 Planned feature improvements
 ============================
 
-* Before running ``pg_start_backup()``, a sanity check that there is a
-  a working ssh connection to the destination would help find
-  configuration errors before disturbing the database.
+* Use 'primary' instead of 'master' in documentation and log output
+  for consistency with PostgreSQL documentation. See also commit
+  870b0a53b627eeb9aca1fc14cbafe25b5beafe12.
+
+* A better check which standby did receive most of the data
+
+* Make the fact that a standby may be delayed a factor in the voting
+  algorithm
+
+* include support for delayed standbys
+
+* Create the repmgr user/database on "master register".
+
+* Use pg_basebackup for the data directory, and ALSO rsync for the
+  configuration files.
+
+* If no configuration file supplied, search in sensible default locations
+  (currently: current directory and `pg_config --sysconfdir`); if
+  possible this should include the location provided by the package,
+  if installed.
+
+* repmgrd: if connection to the upstream node fails on startup, optionally
+  retry for a certain period before giving up; this will cover cases when
+  e.g. primary and standby are both starting up, and the standby comes up
+  before the primary. See github issue #80.
+
+* make old master node ID available for event notification commands
+  (See github issue #80).
+
+* repmgr standby clone: possibility to use barman instead of performing a new base backup
+
+* possibility to transform a failed master into a new standby with pg_rewind
+
+* "repmgr standby switchover" to promote a standby in a controlled manner
+   and convert the existing primary into a standby
+
+* make repmgrd more robust
+
+* repmgr: when cloning a standby using pg_basebackup and replication slots are
+  requested, activate the replication slot using pg_receivexlog to negate the
+  need to set `wal_keep_segments` just for the initial clone (9.4 and 9.5).
+
+* repmgr: enable "standby follow" to point a standby at another standby, not
+  just the replication cluster master (see GitHub #130)
+
+
+Usability improvements
+======================
+
+* repmgr: add interrupt handler, so that if the program is interrupted
+  while running a backup, an attempt can be made to execute pg_stop_backup()
+  on the primary, to prevent an orphaned backup state existing.
+
+* repmgr: when unregistering a node, delete any entries in the repl_monitoring
+  table.
+
+* repmgr: for "standby unregister", accept connection parameters for the
+  primary and perform metadata updates (and slot removal) directly on
+  the primary, to allow a shutdown standby to be unregistered
+  (currently the standby must still be running, which means the replication
+  slot can't be dropped).

check_dir.c

@@ -1,6 +1,6 @@
 /*
  * check_dir.c - Directories management functions
- * Copyright (C) 2ndQuadrant, 2010-2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -23,15 +23,18 @@
 #include <errno.h>
 #include <stdio.h>
 #include <string.h>
+#include <ftw.h>
 
 /* NB: postgres_fe must be included BEFORE check_dir */
-#include "postgres_fe.h"
-#include "check_dir.h"
+#include <libpq-fe.h>
+#include <postgres_fe.h>
 
+#include "check_dir.h"
 #include "strutil.h"
 #include "log.h"
 
-static int mkdir_p(char *path, mode_t omode);
+static bool _create_pg_dir(char *dir, bool force, bool for_witness);
+static int unlink_dir_callback(const char *fpath, const struct stat *sb, int typeflag, struct FTW *ftwbuf);
 
 /*
  * make sure the directory either doesn't exist or is empty
@@ -46,9 +49,9 @@ static int mkdir_p(char *path, mode_t omode);
 int
 check_dir(char *dir)
 {
-	DIR        *chkdir;
-	struct 		dirent *file;
-	int         result = 1;
+	DIR		   *chkdir;
+	struct dirent *file;
+	int			result = 1;
 
 	errno = 0;
 
@@ -60,7 +63,7 @@ check_dir(char *dir)
 	while ((file = readdir(chkdir)) != NULL)
 	{
 		if (strcmp(".", file->d_name) == 0 ||
-		        strcmp("..", file->d_name) == 0)
+			strcmp("..", file->d_name) == 0)
 		{
 			/* skip this and parent directory */
 			continue;
@@ -73,6 +76,7 @@ check_dir(char *dir)
 	}
 
 #ifdef WIN32
+
 	/*
 	 * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
 	 * released version
@@ -84,29 +88,29 @@ check_dir(char *dir)
 	closedir(chkdir);
 
 	if (errno != 0)
-		return -1;          /* some kind of I/O error? */
+		return -1;				/* some kind of I/O error? */
 
 	return result;
 }
 
 
 /*
- * Create directory
+ * Create directory with error log message when failing
  */
 bool
-create_directory(char *dir)
+create_dir(char *dir)
 {
 	if (mkdir_p(dir, 0700) == 0)
 		return true;
 
-	log_err(_("Could not create directory \"%s\": %s\n"),
-	        dir, strerror(errno));
+	log_err(_("unable to create directory \"%s\": %s\n"),
+			dir, strerror(errno));
 
 	return false;
 }
 
 bool
-set_directory_permissions(char *dir)
+set_dir_permissions(char *dir)
 {
 	return (chmod(dir, 0700) != 0) ? false : true;
 }
@@ -124,15 +128,15 @@ set_directory_permissions(char *dir)
  * note that on failure, the path arg has been modified to show the particular
  * directory level we had problems with.
  */
-static int
+int
 mkdir_p(char *path, mode_t omode)
 {
 	struct stat sb;
 	mode_t		numask,
-	oumask;
+				oumask;
 	int			first,
-	last,
-	retval;
+				last,
+				retval;
 	char	   *p;
 
 	p = path;
@@ -151,8 +155,8 @@ mkdir_p(char *path, mode_t omode)
 				return 1;
 		}
 		else if (p[1] == ':' &&
-		         ((p[0] >= 'a' && p[0] <= 'z') ||
-		          (p[0] >= 'A' && p[0] <= 'Z')))
+				 ((p[0] >= 'a' && p[0] <= 'z') ||
+				  (p[0] >= 'A' && p[0] <= 'Z')))
 		{
 			/* local drive */
 			p += 2;
@@ -223,10 +227,123 @@ bool
 is_pg_dir(char *dir)
 {
 	const size_t buf_sz = 8192;
-	char		 path[buf_sz];
-	struct stat	 sb;
+	char		path[buf_sz];
+	struct stat sb;
+	int			r;
 
+	/* test pgdata */
 	xsnprintf(path, buf_sz, "%s/PG_VERSION", dir);
+	if (stat(path, &sb) == 0)
+		return true;
 
-	return (stat(path, &sb) == 0) ? true : false;
+	/* test tablespace dir */
+	sprintf(path, "ls %s/PG_*/ -I*", dir);
+	r = system(path);
+	if (r == 0)
+		return true;
+
+	return false;
+}
+
+
+bool
+create_pg_dir(char *dir, bool force)
+{
+	return _create_pg_dir(dir, force, false);
+}
+
+bool
+create_witness_pg_dir(char *dir, bool force)
+{
+	return _create_pg_dir(dir, force, true);
+}
+
+
+static bool
+_create_pg_dir(char *dir, bool force, bool for_witness)
+{
+	bool		pg_dir = false;
+
+	/* Check this directory could be used as a PGDATA dir */
+	switch (check_dir(dir))
+	{
+		case 0:
+			/* dir not there, must create it */
+			log_info(_("creating directory \"%s\"...\n"), dir);
+
+			if (!create_dir(dir))
+			{
+				log_err(_("unable to create directory \"%s\"...\n"),
+						dir);
+				return false;
+			}
+			break;
+		case 1:
+			/* Present but empty, fix permissions and use it */
+			log_info(_("checking and correcting permissions on existing directory %s ...\n"),
+					 dir);
+
+			if (!set_dir_permissions(dir))
+			{
+				log_err(_("unable to change permissions of directory \"%s\": %s\n"),
+						dir, strerror(errno));
+				return false;
+			}
+			break;
+		case 2:
+			/* Present and not empty */
+			log_warning(_("directory \"%s\" exists but is not empty\n"),
+						dir);
+
+			pg_dir = is_pg_dir(dir);
+
+
+			if (pg_dir && force)
+			{
+
+				/*
+				 * The witness server does not store any data other than a copy of the
+				 * repmgr metadata, so in --force mode we can simply overwrite the
+				 * directory.
+				 *
+				 * For non-witness servers, we'll leave the data in place, both to reduce
+				 * the risk of unintentional data loss and to make it possible for the
+				 * data directory to be brought up-to-date with rsync.
+				 */
+				if (for_witness)
+				{
+					log_notice(_("deleting existing data directory \"%s\"\n"), dir);
+					nftw(dir, unlink_dir_callback, 64, FTW_DEPTH | FTW_PHYS);
+				}
+				/* Let it continue */
+				break;
+			}
+			else if (pg_dir && !force)
+			{
+				log_hint(_("This looks like a PostgreSQL directory.\n"
+							"If you are sure you want to clone here, "
+							"please check there is no PostgreSQL server "
+							"running and use the -F/--force option\n"));
+				return false;
+			}
+
+			return false;
+		default:
+			/* Trouble accessing directory */
+			log_err(_("could not access directory \"%s\": %s\n"),
+					dir, strerror(errno));
+			return false;
+	}
+	return true;
+}
+
+static int
+unlink_dir_callback(const char *fpath, const struct stat *sb, int typeflag, struct FTW *ftwbuf)
+{
+    int rv = remove(fpath);
+
+    if (rv)
+        perror(fpath);
+
+    return rv;
 }

check_dir.h

@@ -1,6 +1,6 @@
 /*
  * check_dir.h
- * Copyright (c) 2ndQuadrant, 2010-2011
+ * Copyright (c) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -20,9 +20,12 @@
 #ifndef _REPMGR_CHECK_DIR_H_
 #define _REPMGR_CHECK_DIR_H_
 
-int  check_dir(char *dir);
-bool create_directory(char *dir);
-bool set_directory_permissions(char *dir);
-bool is_pg_dir(char *dir);
+int			mkdir_p(char *path, mode_t omode);
+int			check_dir(char *dir);
+bool		create_dir(char *dir);
+bool		set_dir_permissions(char *dir);
+bool		is_pg_dir(char *dir);
+bool		create_pg_dir(char *dir, bool force);
+bool		create_witness_pg_dir(char *dir, bool force);
 
 #endif

config.h

@@ -1,6 +1,7 @@
 /*
  * config.h
- * Copyright (c) 2ndQuadrant, 2010-2011
+ *
+ * Copyright (c) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -20,21 +21,124 @@
 #ifndef _REPMGR_CONFIG_H_
 #define _REPMGR_CONFIG_H_
 
-#include "repmgr.h"
+#include "postgres_fe.h"
+
 #include "strutil.h"
 
+#define CONFIG_FILE_NAME	"repmgr.conf"
+
+typedef struct EventNotificationListCell
+{
+	struct EventNotificationListCell *next;
+	char event_type[MAXLEN];
+} EventNotificationListCell;
+
+typedef struct EventNotificationList
+{
+	EventNotificationListCell *head;
+	EventNotificationListCell *tail;
+} EventNotificationList;
+
+typedef struct TablespaceListCell
+{
+	struct TablespaceListCell *next;
+	char		old_dir[MAXPGPATH];
+	char		new_dir[MAXPGPATH];
+} TablespaceListCell;
+
+typedef struct TablespaceList
+{
+	TablespaceListCell *head;
+	TablespaceListCell *tail;
+} TablespaceList;
+
 typedef struct
 {
-	char cluster_name[MAXLEN];
-	int node;
-	char conninfo[MAXLEN];
-	char loglevel[MAXLEN];
-	char logfacility[MAXLEN];
-	char rsync_options[QUERY_STR_LEN];
-} t_configuration_options;
+	char		cluster_name[MAXLEN];
+	int			node;
+	int         upstream_node;
+	char		conninfo[MAXLEN];
+	char		barman_server[MAXLEN];
+	char		barman_config[MAXLEN];
+	int			failover;
+	int			priority;
+	char		node_name[MAXLEN];
+	/* commands executed by repmgrd */
+	char		promote_command[MAXLEN];
+	char		follow_command[MAXLEN];
+	/* Overrides for pg_ctl commands */
+	char		service_stop_command[MAXLEN];
+	char		service_start_command[MAXLEN];
+	char		service_restart_command[MAXLEN];
+	char		service_reload_command[MAXLEN];
+	char		service_promote_command[MAXLEN];
+	char		loglevel[MAXLEN];
+	char		logfacility[MAXLEN];
+	char		rsync_options[QUERY_STR_LEN];
+	char		ssh_options[QUERY_STR_LEN];
+	int			master_response_timeout;
+	int			reconnect_attempts;
+	int			reconnect_interval;
+	char		pg_bindir[MAXLEN];
+	char		pg_ctl_options[MAXLEN];
+	char		pg_basebackup_options[MAXLEN];
+	char		restore_command[MAXLEN];
+	char		logfile[MAXLEN];
+	int			monitor_interval_secs;
+	int			retry_promote_interval_secs;
+	int			witness_repl_nodes_sync_interval_secs;
+	int			use_replication_slots;
+	char		event_notification_command[MAXLEN];
+	EventNotificationList event_notifications;
+	TablespaceList tablespace_mapping;
+}	t_configuration_options;
 
-void parse_config(const char* config_file, t_configuration_options* options);
-void parse_line(char *buff, char *name, char *value);
-char *trim(char *s);
+/*
+ * The following will initialize the structure with a minimal set of options;
+ * actual defaults are set in parse_config() before parsing the configuration file
+ */
+#define T_CONFIGURATION_OPTIONS_INITIALIZER { "", -1, NO_UPSTREAM_NODE, "", "", "", MANUAL_FAILOVER, -1, "", "", "", "", "", "", "", "", "", "", "", "", -1, -1, -1, "", "", "", "", "", 0, 0, 0, 0, "", { NULL, NULL }, { NULL, NULL } }
 
+typedef struct ItemListCell
+{
+	struct ItemListCell *next;
+	char			    *string;
+} ItemListCell;
+
+typedef struct ItemList
+{
+	ItemListCell *head;
+	ItemListCell *tail;
+} ItemList;
+
+typedef struct TablespaceDataListCell
+{
+	struct TablespaceDataListCell *next;
+	char	   *name;
+	char	   *oid;
+	char	   *location;
+	/* optional payload */
+	FILE       *f;
+} TablespaceDataListCell;
+
+typedef struct TablespaceDataList
+{
+	TablespaceDataListCell *head;
+	TablespaceDataListCell *tail;
+} TablespaceDataList;
+
+void set_progname(const char *argv0);
+const char * progname(void);
+
+bool		load_config(const char *config_file, bool verbose, t_configuration_options *options, char *argv0);
+bool		reload_config(t_configuration_options *orig_options);
+bool		parse_config(t_configuration_options *options);
+void		parse_line(char *buff, char *name, char *value);
+char	   *trim(char *s);
+void		item_list_append(ItemList *item_list, char *error_message);
+int			repmgr_atoi(const char *s,
+						const char *config_item,
+						ItemList *error_list,
+						bool allow_negative);
+extern bool		config_file_found;
 #endif

dbutils.h

@@ -1,6 +1,7 @@
 /*
  * dbutils.h
- * Copyright (c) 2ndQuadrant, 2010-2011
+ *
+ * Copyright (c) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -20,16 +21,121 @@
 #ifndef _REPMGR_DBUTILS_H_
 #define _REPMGR_DBUTILS_H_
 
-PGconn *establishDBConnection(const char *conninfo, const bool exit_on_error);
-PGconn *establishDBConnectionByParams(const char *keywords[],
-                                      const char *values[],
-                                      const bool exit_on_error);
-bool	is_standby(PGconn *conn);
-char   *pg_version(PGconn *conn, char* major_version);
-bool	guc_setted(PGconn *conn, const char *parameter, const char *op,
-                const char *value);
-const char	 *get_cluster_size(PGconn *conn);
-PGconn *getMasterConnection(PGconn *standby_conn, int id, char *cluster,
-                            int *master_id, char *master_conninfo_out);
+#include "access/xlogdefs.h"
+#include "pqexpbuffer.h"
 
+#include "config.h"
+#include "strutil.h"
+
+
+typedef enum {
+	UNKNOWN = 0,
+	MASTER,
+	STANDBY,
+	WITNESS
+} t_server_type;
+
+/*
+ * Struct to store node information
+ */
+typedef struct s_node_info
+{
+	int			  node_id;
+	int			  upstream_node_id;
+	t_server_type type;
+	char		  name[MAXLEN];
+	char		  conninfo_str[MAXLEN];
+	char		  slot_name[MAXLEN];
+	int			  priority;
+	bool		  active;
+	bool		  is_ready;
+	bool		  is_visible;
+	XLogRecPtr	  xlog_location;
+}	t_node_info;
+
+
+#define T_NODE_INFO_INITIALIZER { \
+  NODE_NOT_FOUND, \
+  NO_UPSTREAM_NODE, \
+  UNKNOWN, \
+  "", \
+  "", \
+  "", \
+  DEFAULT_PRIORITY, \
+  true, \
+  false, \
+  false, \
+  InvalidXLogRecPtr \
+}
+
+/*
+ * Struct to store replication slot information
+ */
+
+typedef struct s_replication_slot
+{
+	char slot_name[MAXLEN];
+    char slot_type[MAXLEN];
+	bool active;
+}   t_replication_slot;
+
+extern char		repmgr_schema[MAXLEN];
+
+PGconn *_establish_db_connection(const char *conninfo,
+								 const bool exit_on_error,
+								 const bool log_notice,
+								 const bool verbose_only);
+PGconn *establish_db_connection(const char *conninfo,
+								const bool exit_on_error);
+PGconn *establish_db_connection_quiet(const char *conninfo);
+PGconn *test_db_connection(const char *conninfo);
+PGconn *establish_db_connection_by_params(const char *keywords[],
+								  const char *values[],
+								  const bool exit_on_error);
+bool		begin_transaction(PGconn *conn);
+bool		commit_transaction(PGconn *conn);
+bool		rollback_transaction(PGconn *conn);
+bool		check_cluster_schema(PGconn *conn);
+int			is_standby(PGconn *conn);
+bool		is_pgup(PGconn *conn, int timeout);
+int			get_master_node_id(PGconn *conn, char *cluster);
+int			get_server_version(PGconn *conn, char *server_version);
+bool		get_cluster_size(PGconn *conn, char *size);
+bool		get_pg_setting(PGconn *conn, const char *setting, char *output);
+
+int			guc_set(PGconn *conn, const char *parameter, const char *op,
+			const char *value);
+int			guc_set_typed(PGconn *conn, const char *parameter, const char *op,
+			  const char *value, const char *datatype);
+bool		get_conninfo_value(const char *conninfo, const char *keyword, char *output);
+PGconn     *get_upstream_connection(PGconn *standby_conn, char *cluster,
+									int node_id,
+									int *upstream_node_id_ptr,
+									char *upstream_conninfo_out);
+PGconn	   *get_master_connection(PGconn *standby_conn, char *cluster,
+					  int *master_id, char *master_conninfo_out);
+
+int			wait_connection_availability(PGconn *conn, long long timeout);
+bool		cancel_query(PGconn *conn, int timeout);
+char       *get_repmgr_schema(void);
+char       *get_repmgr_schema_quoted(PGconn *conn);
+bool		create_replication_slot(PGconn *conn, char *slot_name, int server_version_num, PQExpBufferData *error_msg);
+int			get_slot_record(PGconn *conn, char *slot_name, t_replication_slot *record);
+bool		drop_replication_slot(PGconn *conn, char *slot_name);
+bool		start_backup(PGconn *conn, char *first_wal_segment, bool fast_checkpoint);
+bool		stop_backup(PGconn *conn, char *last_wal_segment);
+bool		set_config_bool(PGconn *conn, const char *config_param, bool state);
+bool		witness_copy_node_records(PGconn *masterconn, PGconn *witnessconn, char *cluster_name);
+bool		create_node_record(PGconn *conn, char *action, int node, char *type, int upstream_node, char *cluster_name, char *node_name, char *conninfo, int priority, char *slot_name, bool active);
+bool		delete_node_record(PGconn *conn, int node, char *action);
+int			get_node_record(PGconn *conn, char *cluster, int node_id, t_node_info *node_info);
+int			get_node_record_by_name(PGconn *conn, char *cluster, const char *node_name, t_node_info *node_info);
+bool        update_node_record_status(PGconn *conn, char *cluster_name, int this_node_id, char *type, int upstream_node_id, bool active);
+bool        update_node_record_set_upstream(PGconn *conn, char *cluster_name, int this_node_id, int new_upstream_node_id);
+bool        create_event_record(PGconn *conn, t_configuration_options *options, int node_id, char *event, bool successful, char *details);
+
+int		    get_node_replication_state(PGconn *conn, char *node_name, char *output);
+t_server_type parse_node_type(const char *type);
+int			get_data_checksum_version(const char *data_directory);
 #endif
+

debian/DEBIAN/control

@@ -1,9 +1,9 @@
-Package: repmgr
-Version: 1.0-1
+Package: repmgr-auto
+Version: 3.2dev
 Section: database
 Priority: optional
 Architecture: all
-Depends: rsync, postgresql-9.0
-Maintainer: Greg Smith <greg@2ndQuadrant.com>
+Depends: rsync, postgresql-9.3 | postgresql-9.4 | postgresql-9.5
+Maintainer: Self built package <user@localhost>
 Description: PostgreSQL replication setup, magament and monitoring
  has two main executables

debian/repmgr.repmgrd.default

@@ -0,0 +1,18 @@
+# default settings for repmgrd. This file is source by /bin/sh from
+# /etc/init.d/repmgrd
+
+# disable repmgrd by default so it won't get started upon installation
+# valid values: yes/no
+REPMGRD_ENABLED=no
+
+# Options for repmgrd (required)
+#REPMGRD_OPTS="--config-file /path/to/repmgr.conf"
+
+# User to run repmgrd as
+#REPMGRD_USER=postgres
+
+# repmgrd binary
+#REPMGRD_BIN=/usr/bin/repmgrd
+
+# pid file
+#REPMGRD_PIDFILE=/var/run/repmgrd.pid

debian/repmgr.repmgrd.init

@@ -0,0 +1,101 @@
+#!/bin/sh
+### BEGIN INIT INFO
+# Provides:	     repmgrd
+# Required-Start:    $local_fs $remote_fs $network $syslog postgresql
+# Required-Stop:     $local_fs $remote_fs $network $syslog postgresql
+# Should-Start:	     $syslog postgresql
+# Default-Start:     2 3 4 5
+# Default-Stop:	     0 1 6
+# Short-Description: Start/stop repmgrd
+# Description:       Enable repmgrd replication management and monitoring daemon for PostgreSQL
+### END INIT INFO
+
+set -e
+
+DESC="PostgreSQL replication management and monitoring daemon"
+NAME=repmgrd
+
+REPMGRD_ENABLED=no
+REPMGRD_OPTS=
+REPMGRD_USER=postgres
+REPMGRD_BIN=/usr/bin/repmgrd
+REPMGRD_PIDFILE=/var/run/repmgrd.pid
+
+# Read configuration variable file if it is present
+[ -r /etc/default/$NAME ] && . /etc/default/$NAME
+
+test -x $REPMGRD_BIN || exit 0
+
+case "$REPMGRD_ENABLED" in
+    [Yy]*)
+	break
+	;;
+    *)
+	exit 0
+	;;
+esac
+
+# Define LSB log_* functions.
+. /lib/lsb/init-functions
+
+if [ -z "$REPMGRD_OPTS" ]
+then
+    log_warning_msg "Not starting $NAME, REPMGRD_OPTS not set in /etc/default/$NAME"
+    exit 0
+fi
+
+do_start()
+{
+	# Return
+	#   0 if daemon has been started
+	#   1 if daemon was already running
+	#   other if daemon could not be started or a failure occured
+	start-stop-daemon --start --quiet --background --chuid $REPMGRD_USER --make-pidfile --pidfile $REPMGRD_PIDFILE --exec $REPMGRD_BIN -- $REPMGRD_OPTS
+}
+
+do_stop()
+{
+	# Return
+	#   0 if daemon has been stopped
+	#   1 if daemon was already stopped
+	#   other if daemon could not be stopped or a failure occurred
+	start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $REPMGRD_PIDFILE --name "$(basename $REPMGRD_BIN)"
+}
+
+case "$1" in
+  start)
+	log_daemon_msg "Starting $DESC" "$NAME"
+	do_start
+	case "$?" in
+		0) log_end_msg 0 ;;
+		1) log_progress_msg "already started"
+		   log_end_msg 0 ;;
+		*) log_end_msg 1 ;;
+	esac
+
+	;;
+  stop)
+	log_daemon_msg "Stopping $DESC" "$NAME"
+	do_stop
+	case "$?" in
+		0) log_end_msg 0 ;;
+		1) log_progress_msg "already stopped"
+		   log_end_msg 0 ;;
+		*) log_end_msg 1 ;;
+	esac
+
+	;;
+  restart|force-reload)
+	$0 stop
+	$0 start
+	;;
+  status)
+	status_of_proc -p $REPMGRD_PIDFILE $REPMGRD_BIN $NAME && exit 0 || exit $?
+	;;
+  *)
+	echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload|status}" >&2
+	exit 3
+	;;
+esac
+
+exit 0

dirmod.c

@@ -0,0 +1,194 @@
+/*
+ *
+ * dirmod.c
+ *	  directory handling functions
+ *
+ * Copyright (C) 2ndQuadrant, 2010-2016
+ *
+ * Portions Copyright (c) 1996-2013, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "postgres_fe.h"
+
+/* Don't modify declarations in system headers */
+
+#include <unistd.h>
+#include <dirent.h>
+#include <sys/stat.h>
+
+/*
+ * pgfnames
+ *
+ * return a list of the names of objects in the argument directory.  Caller
+ * must call pgfnames_cleanup later to free the memory allocated by this
+ * function.
+ */
+char	  **
+pgfnames(const char *path)
+{
+	DIR		   *dir;
+	struct dirent *file;
+	char	  **filenames;
+	int			numnames = 0;
+	int			fnsize = 200;	/* enough for many small dbs */
+
+	dir = opendir(path);
+	if (dir == NULL)
+	{
+		return NULL;
+	}
+
+	filenames = (char **) palloc(fnsize * sizeof(char *));
+
+	while (errno = 0, (file = readdir(dir)) != NULL)
+	{
+		if (strcmp(file->d_name, ".") != 0 && strcmp(file->d_name, "..") != 0)
+		{
+			if (numnames + 1 >= fnsize)
+			{
+				fnsize *= 2;
+				filenames = (char **) repalloc(filenames,
+											   fnsize * sizeof(char *));
+			}
+			filenames[numnames++] = pstrdup(file->d_name);
+		}
+	}
+
+	if (errno)
+	{
+		fprintf(stderr, _("could not read directory \"%s\": %s\n"),
+				path, strerror(errno));
+	}
+
+	filenames[numnames] = NULL;
+
+	if (closedir(dir))
+	{
+		fprintf(stderr, _("could not close directory \"%s\": %s\n"),
+				path, strerror(errno));
+	}
+
+	return filenames;
+}
+
+
+/*
+ *	pgfnames_cleanup
+ *
+ *	deallocate memory used for filenames
+ */
+void
+pgfnames_cleanup(char **filenames)
+{
+	char	  **fn;
+
+	for (fn = filenames; *fn; fn++)
+		pfree(*fn);
+
+	pfree(filenames);
+}
+
+
+/*
+ *	rmtree
+ *
+ *	Delete a directory tree recursively.
+ *	Assumes path points to a valid directory.
+ *	Deletes everything under path.
+ *	If rmtopdir is true deletes the directory too.
+ *	Returns true if successful, false if there was any problem.
+ *	(The details of the problem are reported already, so caller
+ *	doesn't really have to say anything more, but most do.)
+ */
+bool
+rmtree(const char *path, bool rmtopdir)
+{
+	bool		result = true;
+	char		pathbuf[MAXPGPATH];
+	char	  **filenames;
+	char	  **filename;
+	struct stat statbuf;
+
+	/*
+	 * we copy all the names out of the directory before we start modifying
+	 * it.
+	 */
+	filenames = pgfnames(path);
+
+	if (filenames == NULL)
+		return false;
+
+	/* now we have the names we can start removing things */
+	for (filename = filenames; *filename; filename++)
+	{
+		snprintf(pathbuf, MAXPGPATH, "%s/%s", path, *filename);
+
+		/*
+		 * It's ok if the file is not there anymore; we were just about to
+		 * delete it anyway.
+		 *
+		 * This is not an academic possibility. One scenario where this
+		 * happens is when bgwriter has a pending unlink request for a file in
+		 * a database that's being dropped. In dropdb(), we call
+		 * ForgetDatabaseFsyncRequests() to flush out any such pending unlink
+		 * requests, but because that's asynchronous, it's not guaranteed that
+		 * the bgwriter receives the message in time.
+		 */
+		if (lstat(pathbuf, &statbuf) != 0)
+		{
+			if (errno != ENOENT)
+			{
+				result = false;
+			}
+			continue;
+		}
+
+		if (S_ISDIR(statbuf.st_mode))
+		{
+			/* call ourselves recursively for a directory */
+			if (!rmtree(pathbuf, true))
+			{
+				/* we already reported the error */
+				result = false;
+			}
+		}
+		else
+		{
+			if (unlink(pathbuf) != 0)
+			{
+				if (errno != ENOENT)
+				{
+					result = false;
+				}
+			}
+		}
+	}
+
+	if (rmtopdir)
+	{
+		if (rmdir(path) != 0)
+		{
+			result = false;
+		}
+	}
+
+	pgfnames_cleanup(filenames);
+
+	return result;
+}
+

dirmod.h

@@ -0,0 +1,23 @@
+/*
+ * dirmod.h
+ * Copyright (c) 2ndQuadrant, 2010-2016
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef _DIRMOD_H_
+#define _DIRMOD_H_
+
+#endif

docs/repmgrd-failover-mechanism.md

@@ -0,0 +1,75 @@
+repmgrd's failover algorithm
+============================
+
+When implementing automatic failover, there are two factors which are critical in
+ensuring the desired result is achieved:
+
+  - has the master node genuinely failed?
+  - which is the best node to promote to the new master?
+
+This document outlines repmgrd's decision-making process during automatic failover
+for standbys directly connected to the master node.
+
+
+Master node failure detection
+-----------------------------
+
+If a `repmgrd` instance running on a PostgreSQL standby node is unable to connect to
+the master node, this doesn't neccesarily mean that the master is down and a
+failover is required. Factors such as network connectivity issues could mean that
+even though the standby node is isolated, the replication cluster as a whole
+is functioning correctly, and promoting the standby without further verification
+could result in a "split-brain" situation.
+
+In the event that `repmgrd` is unable to connect to the master node, it will attempt
+to reconnect to the master server several times (as defined by the `reconnect_attempts`
+parameter in `repmgr.conf`), with reconnection attempts  occuring at the interval
+specified by `reconnect_interval`. This happens to verify that the master is definitively
+not accessible (e.g. that connection was not lost due to a brief network glitch).
+
+Appropriate values for these settings will depend very much on the replication
+cluster environment. There will necessarily be a trade-off between the time it
+takes to assume the master is not reachable, and the reliability of that conclusion.
+A standby in a different physical location to the master will probably need a longer
+check interval to rule out possible network issues, whereas one located in the same
+rack with a direct connection between servers could perform the check very quickly.
+
+Note that it's possible the master comes back online after this point is reached,
+but before a new master has been selected; in this case it will be noticed
+during the selection of a new master and no actual failover will take place.
+
+Promotion candidate selection
+-----------------------------
+
+Once `repmgrd` has decided the master is definitively unreachable, following checks
+will be carried out:
+
+* attempts to connect to all other nodes in the cluster (including the witness
+  node, if defined) to establish the state of the cluster, including their
+  current LSN
+
+* If less than half of the nodes are visible (from the viewpoint
+  of this node), `repmgrd` will not take any further action. This is to ensure that
+  e.g. if a replication cluster is spread over multiple data centres, a split-brain
+  situation does not occur if there is a network failure between datacentres. Note
+  that if nodes are split evenly between data centres, a witness server can be
+  used to establish the "majority" daat centre.
+
+* `repmgrd` polls all visible servers and waits for each node to return a valid LSN;
+  it updates the LSN previously  stored for this node if it has increased since
+  the initial check
+
+* once all LSNs have been retrieved, `repmgrd` will check for the highest LSN; if
+  its own node has the highest LSN, it will attempt to promote itself (using the
+  command defined in `promote_command` in `repmgr.conf`. Note that if using
+  `repmgr standby promote` as the promotion command, and the original master becomes available
+  before the promotion takes effect, `repmgr` will return an error and no promotion
+  will take place, and `repmgrd` will resume monitoring as usual.
+
+* if the node is not the promotion candidate, `repmgrd` will execute the
+  `follow_command` defined in `repmgr.conf`. If using `repmgr standby follow` here,
+  `repmgr` will attempt to detect the new master node and attach to that.
+
+
+
+

docs/repmgrd-node-fencing.md

@@ -0,0 +1,150 @@
+Fencing a failed master node with repmgrd and pgbouncer
+=======================================================
+
+With automatic failover, it's essential to ensure that a failed master
+remains inaccessible to your application, even if it comes back online
+again, to avoid a split-brain situation.
+
+By using `pgbouncer` together with `repmgrd`, it's possible to combine
+automatic failover with a process to isolate the failed master from
+your application and ensure that all connections which should go to
+the master are directed there smoothly without having to reconfigure
+your application. (Note that as a connection pooler, `pgbouncer` can
+benefit your application in other ways, but those are beyond the scope
+of this document).
+
+* * *
+
+> *WARNING*: automatic failover is tricky to get right. This document
+> demonstrates one possible implementation method, however you should
+> carefully configure and test any setup to suit the needs of your own
+> replication cluster/application.
+
+* * *
+
+In a failover situation, `repmgrd` promotes a standby to master by
+executing the command defined in `promote_command`. Normally this
+would be something like:
+
+    repmgr standby promote -f /etc/repmgr.conf
+
+By wrapping this in a custom script which adjusts the `pgbouncer`
+configuration on all nodes, it's possible to fence the failed master
+and redirect write connections to the new master.
+
+The script consists of three sections:
+
+* commands to pause `pgbouncer` on all nodes
+* the promotion command itself
+* commands to reconfigure and restart `pgbouncer` on all nodes
+
+Note that it requires password-less SSH access between all nodes to be
+able to update the `pgbouncer` configuration files.
+
+For the purposes of this demonstration, we'll assume there are 3 nodes
+(master and two standbys), with `pgbouncer` listening on port 6432
+handling connections to a database called `appdb`. The `postgres`
+system user must have write access to the `pgbouncer` configuration
+file on all nodes, assumed to be at `/etc/pgbouncer.ini`.
+
+The script also requires a template file containing global `pgbouncer`
+configuration, which should looks something like this (adjust
+settings appropriately for your environment):
+
+`/var/lib/postgres/repmgr/pgbouncer.ini.template`
+
+    [pgbouncer]
+
+    logfile = /var/log/pgbouncer/pgbouncer.log
+    pidfile = /var/run/pgbouncer/pgbouncer.pid
+
+    listen_addr = *
+    listen_port = 6532
+    unix_socket_dir = /tmp
+
+    auth_type = trust
+    auth_file = /etc/pgbouncer.auth
+
+    admin_users = postgres
+    stats_users = postgres
+
+    pool_mode = transaction
+
+    max_client_conn = 100
+    default_pool_size = 20
+    min_pool_size = 5
+    reserve_pool_size = 5
+    reserve_pool_timeout = 3
+
+    log_connections = 1
+    log_disconnections = 1
+    log_pooler_errors = 1
+
+The actual script is as follows; adjust the configurable items as appropriate:
+
+`/var/lib/postgres/repmgr/promote.sh`
+
+
+    #!/usr/bin/env bash
+    set -u
+    set -e
+
+    # Configurable items
+    PGBOUNCER_HOSTS="node1 node2 node3"
+    REPMGR_DB="repmgr"
+    REPMGR_USER="repmgr"
+    REPMGR_SCHEMA="repmgr_test"
+    PGBOUNCER_CONFIG="/etc/pgbouncer.ini"
+    PGBOUNCER_INI_TEMPLATE="/var/lib/postgres/repmgr/pgbouncer.ini.template"
+    PGBOUNCER_DATABASE="appdb"
+
+    # 1. Pause running pgbouncer instances
+    for HOST in $PGBOUNCER_HOSTS
+    do
+        psql -t -c "pause" -h $HOST -p $PORT -U postgres pgbouncer
+    done
+
+
+    # 2. Promote this node from standby to master
+
+    repmgr standby promote -f /etc/repmgr.conf
+
+
+    # 3. Reconfigure pgbouncer instances
+
+    PGBOUNCER_INI_NEW="/tmp/pgbouncer.ini.new"
+
+    for HOST in $PGBOUNCER_HOSTS
+    do
+        # Recreate the pgbouncer config file
+        echo -e "[databases]\n" > $PGBOUNCER_INI_NEW
+
+        psql -d $REPMGR_DB -U $REPMGR_USER -t -A \
+          -c "SELECT '$PGBOUNCER_DATABASE= ' || conninfo || ' application_name=pgbouncer_$HOST' \
+              FROM $REPMGR_SCHEMA.repl_nodes \
+              WHERE active = TRUE AND type='master'" >> $PGBOUNCER_INI_NEW
+
+        cat $PGBOUNCER_INI_TEMPLATE >> $PGBOUNCER_INI_NEW
+
+        rsync $PGBOUNCER_INI_NEW $HOST:$PGBOUNCER_CONFIG
+
+        psql -tc "reload" -h $HOST -U postgres pgbouncer
+        psql -tc "resume" -h $HOST -U postgres pgbouncer
+
+    done
+
+    # Clean up generated file
+    rm $PGBOUNCER_INI_NEW
+
+    echo "Reconfiguration of pgbouncer complete"
+
+Script and template file should be installed on each node where
+`repmgrd` is running.
+
+Finally, set `promote_command` in `repmgr.conf` on each node to
+point to the custom promote script:
+
+    promote_command=/var/lib/postgres/repmgr/promote.sh
+
+and reload/restart any running `repmgrd` instances for the changes to take
+effect.

errcode.h

@@ -1,6 +1,6 @@
 /*
  * errcode.h
- * Copyright (C) 2ndQuadrant, 2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -25,13 +25,21 @@
 #define SUCCESS 0
 #define ERR_BAD_CONFIG 1
 #define ERR_BAD_RSYNC 2
-#define ERR_STOP_BACKUP 3
 #define ERR_NO_RESTART 4
-#define ERR_NEEDS_XLOG 5
 #define ERR_DB_CON 6
 #define ERR_DB_QUERY 7
 #define ERR_PROMOTED 8
-#define ERR_BAD_PASSWORD 9
 #define ERR_STR_OVERFLOW 10
+#define ERR_FAILOVER_FAIL 11
+#define ERR_BAD_SSH 12
+#define ERR_SYS_FAILURE 13
+#define ERR_BAD_BASEBACKUP 14
+#define ERR_INTERNAL 15
+#define ERR_MONITORING_FAIL 16
+#define ERR_BAD_BACKUP_LABEL 17
+#define ERR_SWITCHOVER_FAIL 18
+#define ERR_BARMAN 19
+#define ERR_REGISTRATION_SYNC 20
 
-#endif	/* _ERRCODE_H_ */
+
+#endif   /* _ERRCODE_H_ */

log.c

@@ -1,6 +1,6 @@
 /*
  * log.c - Logging methods
- * Copyright (C) 2ndQuadrant, 2010-2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  * This module is a set of methods for logging (currently only syslog)
  *
@@ -25,9 +25,11 @@
 
 #ifdef HAVE_SYSLOG
 #include <syslog.h>
-#include <stdarg.h>
 #endif
 
+#include <stdarg.h>
+#include <time.h>
+
 #include "log.h"
 
 #define DEFAULT_IDENT "repmgr"
@@ -37,20 +39,119 @@
 
 /* #define REPMGR_DEBUG */
 
-static int detect_log_level(const char* level);
-static int detect_log_facility(const char* facility);
+static int	detect_log_facility(const char *facility);
+static void _stderr_log_with_level(const char *level_name, int level, const char *fmt, va_list ap)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 0)));
 
-int log_type = REPMGR_STDERR;
-int log_level = LOG_NOTICE;
+int			log_type = REPMGR_STDERR;
+int			log_level = LOG_NOTICE;
+int			last_log_level = LOG_NOTICE;
+int			verbose_logging = false;
+int			terse_logging = false;
 
-bool logger_init(const char* ident, const char* level, const char* facility)
+extern void
+stderr_log_with_level(const char *level_name, int level, const char *fmt, ...)
 {
+	va_list		arglist;
 
-	int l;
-	int f;
+	va_start(arglist, fmt);
+	_stderr_log_with_level(level_name, level, fmt, arglist);
+	va_end(arglist);
+}
+
+static void
+_stderr_log_with_level(const char *level_name, int level, const char *fmt, va_list ap)
+{
+	time_t		t;
+	struct tm  *tm;
+	char		buff[100];
+
+	/*
+	 * Store the requested level so that if there's a subsequent
+	 * log_hint(), we can suppress that if appropriate.
+	 */
+	last_log_level = level;
+
+	if (log_level >= level)
+	{
+		time(&t);
+		tm = localtime(&t);
+		strftime(buff, 100, "[%Y-%m-%d %H:%M:%S]", tm);
+		fprintf(stderr, "%s [%s] ", buff, level_name);
+
+		vfprintf(stderr, fmt, ap);
+
+		fflush(stderr);
+	}
+}
+
+void
+log_hint(const char *fmt, ...)
+{
+	va_list		ap;
+
+	if (terse_logging == false)
+	{
+		va_start(ap, fmt);
+		_stderr_log_with_level("HINT", last_log_level, fmt, ap);
+		va_end(ap);
+	}
+}
+
+
+void
+log_verbose(int level, const char *fmt, ...)
+{
+	va_list		ap;
+
+	va_start(ap, fmt);
+
+	if (verbose_logging == true)
+	{
+		switch(level)
+		{
+			case LOG_EMERG:
+				_stderr_log_with_level("EMERG", level, fmt, ap);
+				break;
+			case LOG_ALERT:
+				_stderr_log_with_level("ALERT", level, fmt, ap);
+				break;
+			case LOG_CRIT:
+				_stderr_log_with_level("CRIT", level, fmt, ap);
+				break;
+			case LOG_ERR:
+				_stderr_log_with_level("ERR", level, fmt, ap);
+				break;
+			case LOG_WARNING:
+				_stderr_log_with_level("WARNING", level, fmt, ap);
+				break;
+			case LOG_NOTICE:
+				_stderr_log_with_level("NOTICE", level, fmt, ap);
+				break;
+			case LOG_INFO:
+				_stderr_log_with_level("INFO", level, fmt, ap);
+				break;
+			case LOG_DEBUG:
+				_stderr_log_with_level("DEBUG", level, fmt, ap);
+				break;
+		}
+	}
+
+	va_end(ap);
+}
+
+
+bool
+logger_init(t_configuration_options *opts, const char *ident)
+{
+	char	   *level = opts->loglevel;
+	char	   *facility = opts->logfacility;
+
+	int			l;
+	int			f;
 
 #ifdef HAVE_SYSLOG
-	int syslog_facility = DEFAULT_SYSLOG_FACILITY;
+	int			syslog_facility = DEFAULT_SYSLOG_FACILITY;
 #endif
 
 #ifdef REPMGR_DEBUG
@@ -69,10 +170,10 @@ bool logger_init(const char* ident, const char* level, const char* facility)
 		printf("Assigned level for logger: %d\n", l);
 #endif
 
-		if (l > 0)
+		if (l >= 0)
 			log_level = l;
 		else
-			stderr_log_warning(_("Cannot detect log level %s (use any of DEBUG, INFO, NOTICE, WARNING, ERR, ALERT, CRIT or EMERG)\n"), level);
+			stderr_log_warning(_("Invalid log level \"%s\" (available values: DEBUG, INFO, NOTICE, WARNING, ERR, ALERT, CRIT or EMERG)\n"), level);
 	}
 
 	if (facility && *facility)
@@ -107,21 +208,53 @@ bool logger_init(const char* ident, const char* level, const char* facility)
 
 	if (log_type == REPMGR_SYSLOG)
 	{
-		setlogmask (LOG_UPTO (log_level));
-		openlog (ident, LOG_CONS | LOG_PID | LOG_NDELAY, syslog_facility);
+		setlogmask(LOG_UPTO(log_level));
+		openlog(ident, LOG_CONS | LOG_PID | LOG_NDELAY, syslog_facility);
 
 		stderr_log_notice(_("Setup syslog (level: %s, facility: %s)\n"), level, facility);
 	}
-
 #endif
 
-	return true;
+	if (*opts->logfile)
+	{
+		FILE	   *fd;
 
+		/* Check if we can write to the specified file before redirecting
+		 * stderr - if freopen() fails, stderr output will vanish into
+		 * the ether and the user won't know what's going on.
+		 */
+
+		fd = fopen(opts->logfile, "a");
+		if (fd == NULL)
+		{
+			stderr_log_err(_("Unable to open specified logfile '%s' for writing: %s\n"), opts->logfile, strerror(errno));
+			stderr_log_err(_("Terminating\n"));
+			exit(ERR_BAD_CONFIG);
+		}
+		fclose(fd);
+
+		stderr_log_notice(_("Redirecting logging output to '%s'\n"), opts->logfile);
+		fd = freopen(opts->logfile, "a", stderr);
+
+		/* It's possible freopen() may still fail due to e.g. a race condition;
+		   as it's not feasible to restore stderr after a failed freopen(),
+		   we'll write to stdout as a last resort.
+		 */
+		if (fd == NULL)
+		{
+			printf(_("Unable to open specified logfile %s for writing: %s\n"), opts->logfile, strerror(errno));
+			printf(_("Terminating\n"));
+			exit(ERR_BAD_CONFIG);
+		}
+	}
+
+	return true;
 }
 
-bool logger_shutdown(void)
-{
 
+bool
+logger_shutdown(void)
+{
 #ifdef HAVE_SYSLOG
 	if (log_type == REPMGR_SYSLOG)
 		closelog();
@@ -131,17 +264,34 @@ bool logger_shutdown(void)
 }
 
 /*
- * Set a minimum logging level.  Intended for command line verbosity
- * options, which might increase requested logging over what's specified
- * in the regular configuration file.
+ * Indicate whether extra-verbose logging is required. This will
+ * generate a lot of output, particularly debug logging, and should
+ * not be permanently enabled in production.
+ *
+ * NOTE: in previous repmgr versions, this option forced the log
+ * level to INFO.
  */
-void logger_min_verbose(int minimum)
+void
+logger_set_verbose(void)
 {
-	if (log_level < minimum)
-		log_level = minimum;
+	verbose_logging = true;
 }
 
-int detect_log_level(const char* level)
+
+/*
+ * Indicate whether some non-critical log messages can be omitted.
+ * Currently this includes warnings about irrelevant command line
+ * options and hints.
+ */
+
+void logger_set_terse(void)
+{
+	terse_logging = true;
+}
+
+
+int
+detect_log_level(const char *level)
 {
 	if (!strcmp(level, "DEBUG"))
 		return LOG_DEBUG;
@@ -160,43 +310,44 @@ int detect_log_level(const char* level)
 	if (!strcmp(level, "EMERG"))
 		return LOG_EMERG;
 
-	return 0;
+	return -1;
 }
 
-int detect_log_facility(const char* facility)
+static int
+detect_log_facility(const char *facility)
 {
-	int local = 0;
+	int			local = 0;
+
 	if (!strncmp(facility, "LOCAL", 5) && strlen(facility) == 6)
 	{
-
-		local = atoi (&facility[5]);
+		local = atoi(&facility[5]);
 
 		switch (local)
 		{
-		case 0:
-			return LOG_LOCAL0;
-			break;
-		case 1:
-			return LOG_LOCAL1;
-			break;
-		case 2:
-			return LOG_LOCAL2;
-			break;
-		case 3:
-			return LOG_LOCAL3;
-			break;
-		case 4:
-			return LOG_LOCAL4;
-			break;
-		case 5:
-			return LOG_LOCAL5;
-			break;
-		case 6:
-			return LOG_LOCAL6;
-			break;
-		case 7:
-			return LOG_LOCAL7;
-			break;
+			case 0:
+				return LOG_LOCAL0;
+				break;
+			case 1:
+				return LOG_LOCAL1;
+				break;
+			case 2:
+				return LOG_LOCAL2;
+				break;
+			case 3:
+				return LOG_LOCAL3;
+				break;
+			case 4:
+				return LOG_LOCAL4;
+				break;
+			case 5:
+				return LOG_LOCAL5;
+				break;
+			case 6:
+				return LOG_LOCAL6;
+				break;
+			case 7:
+				return LOG_LOCAL7;
+				break;
 		}
 
 	}

log.h

@@ -1,6 +1,6 @@
 /*
  * log.h
- * Copyright (c) 2ndQuadrant, 2010-2011
+ * Copyright (c) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -25,15 +25,19 @@
 #define REPMGR_SYSLOG 1
 #define REPMGR_STDERR 2
 
+extern void
+stderr_log_with_level(const char *level_name, int level, const char *fmt,...)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
+
 /* Standard error logging */
-#define stderr_log_debug(...) if (log_level >= LOG_DEBUG) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_info(...) if (log_level >= LOG_INFO) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_notice(...) if (log_level >= LOG_NOTICE) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_warning(...) if (log_level >= LOG_WARNING) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_err(...) if (log_level >= LOG_ERR) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_crit(...) if (log_level >= LOG_CRIT) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_alert(...) if (log_level >= LOG_ALERT) fprintf(stderr, __VA_ARGS__)
-#define stderr_log_emerg(...) if (log_level >= LOG_EMERG) fprintf(stderr, __VA_ARGS__)
+#define stderr_log_debug(...) stderr_log_with_level("DEBUG", LOG_DEBUG, __VA_ARGS__)
+#define stderr_log_info(...)  stderr_log_with_level("INFO", LOG_INFO, __VA_ARGS__)
+#define stderr_log_notice(...) stderr_log_with_level("NOTICE", LOG_NOTICE, __VA_ARGS__)
+#define stderr_log_warning(...) stderr_log_with_level("WARNING", LOG_WARNING, __VA_ARGS__)
+#define stderr_log_err(...) stderr_log_with_level("ERROR", LOG_ERR, __VA_ARGS__)
+#define stderr_log_crit(...) stderr_log_with_level("CRITICAL", LOG_CRIT, __VA_ARGS__)
+#define stderr_log_alert(...) stderr_log_with_level("ALERT", LOG_ALERT, __VA_ARGS__)
+#define stderr_log_emerg(...) stderr_log_with_level("EMERGENCY", LOG_EMERG, __VA_ARGS__)
 
 #ifdef HAVE_SYSLOG
 
@@ -86,17 +90,16 @@
 		if (log_type == REPMGR_SYSLOG) syslog(LOG_ALERT, __VA_ARGS__); \
 		else stderr_log_alert(__VA_ARGS__); \
 	}
-
 #else
 
-#define	LOG_EMERG	0	/* system is unusable */
-#define	LOG_ALERT	1	/* action must be taken immediately */
-#define	LOG_CRIT	2	/* critical conditions */
-#define	LOG_ERR		3	/* error conditions */
-#define	LOG_WARNING	4	/* warning conditions */
-#define	LOG_NOTICE	5	/* normal but significant condition */
-#define	LOG_INFO	6	/* informational */
-#define	LOG_DEBUG	7	/* debug-level messages */
+#define LOG_EMERG	0			/* system is unusable */
+#define LOG_ALERT	1			/* action must be taken immediately */
+#define LOG_CRIT	2			/* critical conditions */
+#define LOG_ERR		3			/* error conditions */
+#define LOG_WARNING 4			/* warning conditions */
+#define LOG_NOTICE	5			/* normal but significant condition */
+#define LOG_INFO	6			/* informational */
+#define LOG_DEBUG	7			/* debug-level messages */
 
 #define log_debug(...) stderr_log_debug(__VA_ARGS__)
 #define log_info(...) stderr_log_info(__VA_ARGS__)
@@ -106,16 +109,28 @@
 #define log_crit(...) stderr_log_crit(__VA_ARGS__)
 #define log_alert(...) stderr_log_alert(__VA_ARGS__)
 #define log_emerg(...) stderr_log_emerg(__VA_ARGS__)
-
 #endif
 
 
+int			detect_log_level(const char *level);
+
 /* Logger initialisation and shutdown */
-bool logger_shutdown(void);
-bool logger_init(const char* ident, const char* level, const char* facility);
-void logger_min_verbose(int minimum);
 
-extern int log_type;
-extern int log_level;
+bool		logger_init(t_configuration_options * opts, const char *ident);
 
-#endif
+bool		logger_shutdown(void);
+
+void		logger_set_verbose(void);
+void		logger_set_terse(void);
+
+void		log_hint(const char *fmt, ...)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
+void		log_verbose(int level, const char *fmt, ...)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
+
+extern int	log_type;
+extern int	log_level;
+extern int	verbose_logging;
+extern int	terse_logging;
+
+#endif /* _REPMGR_LOG_H_ */

repmgr.conf

@@ -1,21 +0,0 @@
-###################################################
-# Replication Manager configuration file
-###################################################
-
-# Cluster name
-cluster=test
-
-# Node ID
-node=2
-
-# Connection information
-conninfo='host=192.168.204.104'
-rsync_options=--archive --checksum --compress --progress --rsh=ssh
-
-# Log level: possible values are DEBUG, INFO, NOTICE, WARNING, ERR, ALERT, CRIT or EMERG
-# Default: NOTICE
-loglevel=NOTICE
-
-# Logging facility: possible values are STDERR or - for Syslog integration - one of LOCAL0, LOCAL1, ..., LOCAL7, USER
-# Default: STDERR
-logfacility=STDERR

repmgr.conf.sample

@@ -0,0 +1,200 @@
+###################################################
+# Replication Manager sample configuration file
+###################################################
+
+# Some configuration items will be set with a default value; this
+# is noted for each item. Where no default value is shown, the
+# parameter will be treated as empty or false.
+
+# Required configuration items
+# ============================
+#
+# repmgr and repmgrd require these items to be configured:
+
+# Cluster name - this will be used by repmgr to generate its internal
+# schema (pattern: "repmgr_{cluster}"); while this name will be quoted
+# to preserve case, we recommend using lower case and avoiding whitespace
+# to facilitate easier querying of the repmgr views and tables.
+#cluster=example_cluster
+
+# Node ID and name
+# (Note: we recommend to avoid naming nodes after their initial
+#  replication function, as this will cause confusion when e.g.
+#  "standby2" is promoted to primary)
+#node=2           # a unique integer
+#node_name=node2  # an arbitrary (but unique) string; we recommend using
+                 # the server's hostname or another identifier unambiguously
+                 # associated with the server to avoid confusion
+
+# Database connection information as a conninfo string
+# This must be accessible to all servers in the cluster; for details see:
+#
+#   https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
+#
+#conninfo='host=192.168.204.104 dbname=repmgr user=repmgr'
+#
+# If repmgrd is in use, consider explicitly setting `connect_timeout` in the
+# conninfo string to determine the length of time which elapses before
+# a network connection attempt is abandoned; for details see:
+#
+#   https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNECT-CONNECT-TIMEOUT
+
+# Optional configuration items
+# ============================
+
+# Replication settings
+# ---------------------
+
+# When using cascading replication, a standby can connect to another
+# upstream standby node which is specified by setting 'upstream_node'.
+# In that case, the upstream node must exist before the new standby
+# can be registered. If 'upstream_node' is not set, then the standby
+# will connect directly to the primary node.
+#upstream_node=1
+
+# use physical replication slots - PostgreSQL 9.4 and later only
+# (default: 0)
+#use_replication_slots=0
+
+# NOTE: 'max_replication_slots' should be configured for at least the
+# number of standbys which will connect to the primary.
+
+# Logging and monitoring settings
+# -------------------------------
+
+# Log level: possible values are DEBUG, INFO, NOTICE, WARNING, ERR, ALERT, CRIT or EMERG
+# (default: NOTICE)
+#loglevel=NOTICE
+
+# Logging facility: possible values are STDERR or - for Syslog integration - one of LOCAL0, LOCAL1, ..., LOCAL7, USER
+# (default: STDERR)
+#logfacility=STDERR
+
+# stderr can be redirected to an arbitrary file:
+#
+#logfile='/var/log/repmgr/repmgr.log'
+
+# event notifications can be passed to an arbitrary external program
+# together with the following parameters:
+#
+#   %n - node ID
+#   %e - event type
+#   %s - success (1 or 0)
+#   %t - timestamp
+#   %d - details
+#
+# the values provided for "%t" and "%d" will probably contain spaces,
+# so should be quoted in the provided command configuration, e.g.:
+#
+#event_notification_command='/path/to/some/script %n %e %s "%t" "%d"'
+
+# By default, all notifications will be passed; the notification types
+# can be filtered to explicitly named ones:
+#
+#event_notifications=master_register,standby_register,witness_create
+
+
+# Environment/command settings
+# ----------------------------
+
+# path to PostgreSQL binary directory (location of pg_ctl, pg_basebackup etc.)
+# (if not provided, defaults to system $PATH)
+#pg_bindir=/usr/bin/
+#
+# Debian/Ubuntu users: you will probably need to set this to the directory
+# where `pg_ctl` is located, e.g. /usr/lib/postgresql/9.5/bin/
+
+# service control commands
+#
+# repmgr provides options to override the default pg_ctl commands
+# used to stop, start, restart, reload and promote the PostgreSQL cluster
+#
+# NOTE: These commands must be runnable on remote nodes as well for switchover
+# to function correctly.
+#
+# If you use sudo, the user repmgr runs as (usually 'postgres')  must have
+# passwordless sudo access to execute the command
+#
+# For example, to use systemd, you may use the following configuration:
+#
+#    # this is required when running sudo over ssh without -t:
+#    Defaults:postgres !requiretty
+#    postgres ALL = NOPASSWD: /usr/bin/systemctl stop postgresql-9.5, \
+#       /usr/bin/systemctl start postgresql-9.5, \
+#       /usr/bin/systemctl restart postgresql-9.5
+#
+# service_start_command = systemctl start postgresql-9.5
+# service_stop_command = systemctl stop postgresql-9.5
+# service_restart_command = systemctl restart postgresql-9.5
+# service_reload_command = pg_ctlcluster 9.5 main reload
+# service_promote_command = pg_ctlcluster 9.5 main promote
+
+# external command options
+
+#rsync_options=--archive --checksum --compress --progress --rsh="ssh -o \"StrictHostKeyChecking no\""
+#ssh_options=-o "StrictHostKeyChecking no"
+
+# external command arguments. Values shown are examples.
+
+#pg_ctl_options='-s'
+#pg_basebackup_options='--xlog-method=s'
+
+
+# Standby clone settings
+# ----------------------
+#
+# These settings apply when cloning a standby (`repmgr standby clone`).
+
+# Tablespaces can be remapped from one file system location to another:
+#
+# tablespace_mapping=/path/to/original/tablespace=/path/to/new/tablespace
+
+# You can specify a restore_command to be used in the recovery.conf that
+# will be placed in the cloned standby
+#
+# restore_command = cp /path/to/archived/wals/%f %p
+
+# Failover settings (repmgrd)
+# ---------------------------
+#
+# These settings are only applied when repmgrd is running. Values shown
+# are defaults.
+
+# monitoring interval in seconds; default is 2
+#monitor_interval_secs=2
+
+# Number of seconds to wait for a response from the primary server before
+# deciding it has failed.
+
+#master_response_timeout=60
+
+# Number of attempts at what interval (in seconds) to try and
+# connect to  a server to establish its status (e.g. master
+# during failover)
+#reconnect_attempts=6
+#reconnect_interval=10
+
+# Autofailover options
+#failover=manual    # one of 'automatic', 'manual' (default: manual)
+                    # defines the action to take in the event of upstream failure
+                    #
+                    # 'automatic': repmgrd will automatically attempt to promote the
+                    #    node or follow the new upstream node
+                    # 'manual': repmgrd will take no action and the mode will require
+                    #    manual attention to reattach it to replication
+
+#priority=100       # indicate a preferred priorty for promoting nodes
+                    # a value of zero or less prevents the node being promoted to primary
+                    # (default: 100)
+
+#promote_command='repmgr standby promote -f /path/to/repmgr.conf'
+#follow_command='repmgr standby follow -f /path/to/repmgr.conf -W'
+
+# change wait time for primary; before we bail out and exit when the primary
+# disappears, we wait 'reconnect_attempts' * 'retry_promote_interval_secs'
+# seconds; by default this would be half an hour, as 'retry_promote_interval_secs'
+# default value is 300)
+#retry_promote_interval_secs=300
+
+# Number of seconds after which the witness server resyncs the repl_nodes table
+#witness_repl_nodes_sync_interval_secs=15

repmgr.h

@@ -1,6 +1,6 @@
 /*
  * repmgr.h
- * Copyright (c) 2ndQuadrant, 2010-2011
+ * Copyright (c) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -20,44 +20,216 @@
 #ifndef _REPMGR_H_
 #define _REPMGR_H_
 
-#include "postgres_fe.h"
-#include "getopt_long.h"
-#include "libpq-fe.h"
+#include <libpq-fe.h>
+#include <postgres_fe.h>
+#include <getopt_long.h>
+#include "pqexpbuffer.h"
 
 #include "strutil.h"
 #include "dbutils.h"
 #include "errcode.h"
-
-#define PRIMARY_MODE		0
-#define STANDBY_MODE		1
-
 #include "config.h"
-#define MAXFILENAME		1024
-#define ERRBUFF_SIZE		512
+#include "dirmod.h"
 
-#define DEFAULT_CONFIG_FILE		"./repmgr.conf"
-#define DEFAULT_WAL_KEEP_SEGMENTS	"5000"
+#define MIN_SUPPORTED_VERSION		"9.3"
+#define MIN_SUPPORTED_VERSION_NUM	90300
+
+#define ERRBUFF_SIZE	512
+
+#define DEFAULT_WAL_KEEP_SEGMENTS	"0"
 #define DEFAULT_DEST_DIR		"."
-#define DEFAULT_MASTER_PORT		"5432"
-#define DEFAULT_DBNAME			"postgres"
 #define DEFAULT_REPMGR_SCHEMA_PREFIX	"repmgr_"
+#define DEFAULT_PRIORITY		100
+#define FAILOVER_NODES_MAX_CHECK 50
+
+#define MANUAL_FAILOVER		0
+#define AUTOMATIC_FAILOVER	1
+#define NODE_NOT_FOUND      -1
+#define NO_UPSTREAM_NODE	-1
+#define UNKNOWN_NODE_ID     -1
+
+/* command line options without short versions */
+#define OPT_HELP                         1
+#define OPT_CHECK_UPSTREAM_CONFIG        2
+#define OPT_RECOVERY_MIN_APPLY_DELAY     3
+#define OPT_COPY_EXTERNAL_CONFIG_FILES   4
+#define OPT_CONFIG_ARCHIVE_DIR           5
+#define OPT_PG_REWIND                    6
+#define OPT_PWPROMPT                     7
+#define OPT_CSV                          8
+#define OPT_NODE                         9
+#define OPT_WITHOUT_BARMAN               10
+#define OPT_NO_UPSTREAM_CONNECTION       11
+#define OPT_REGISTER_WAIT                12
+#define OPT_CLUSTER                      13
+
+/* deprecated command line options */
+#define OPT_INITDB_NO_PWPROMPT           998
+#define OPT_IGNORE_EXTERNAL_CONFIG_FILES 999
+
+/* values for --copy-external-config-files */
+#define CONFIG_FILE_SAMEPATH 1
+#define CONFIG_FILE_PGDATA 2
+
 
 /* Run time options type */
 typedef struct
 {
+	/* general repmgr options */
+	char		config_file[MAXPGPATH];
+	bool		verbose;
+	bool		terse;
+	bool		force;
 
-	char dbname[MAXLEN];
-	char host[MAXLEN];
-	char username[MAXLEN];
-	char dest_dir[MAXFILENAME];
-	char config_file[MAXFILENAME];
-	char remote_user[MAXLEN];
-	char wal_keep_segments[MAXLEN];
-	bool verbose;
-	bool force;
+	/* options which override setting in repmgr.conf */
+	char		loglevel[MAXLEN];
+	char		pg_bindir[MAXLEN];
+
+	/* connection parameters */
+	char		dbname[MAXLEN];
+	char		host[MAXLEN];
+	char		username[MAXLEN];
+	char		dest_dir[MAXPGPATH];
+	char		remote_user[MAXLEN];
+	char		superuser[MAXLEN];
+	char		masterport[MAXLEN];
+	bool		conninfo_provided;
+	bool		connection_param_provided;
+	bool		host_param_provided;
+
+	/* standby clone parameters */
+	bool		wal_keep_segments_used;
+	char		wal_keep_segments[MAXLEN];
+	bool		ignore_rsync_warn;
+	bool		rsync_only;
+	bool		fast_checkpoint;
+	bool		without_barman;
+	bool		no_upstream_connection;
+	bool		copy_external_config_files;
+	int			copy_external_config_files_destination;
+	bool		wait_register_sync;
+	int			wait_register_sync_seconds;
+
+	char		recovery_min_apply_delay[MAXLEN];
+
+	/* witness create parameters */
+	bool		witness_pwprompt;
+
+	/* standby follow parameters */
+	bool		wait_for_master;
+
+	/* cluster {show|matrix|crosscheck} parameters */
+	bool		csv_mode;
+
+	/* cluster cleanup parameters */
+	int			keep_history;
+
+	/* standby switchover parameters */
+	char		remote_config_file[MAXLEN];
+	bool		pg_rewind_supplied;
+	char		pg_rewind[MAXPGPATH];
+	char		pg_ctl_mode[MAXLEN];
+
+	/* standby {archive_config | restore_config} parameters  */
+	char		config_archive_dir[MAXLEN];
+
+	/* {standby|witness} unregister parameters */
+	int			node;
+
+}	t_runtime_options;
+
+#define T_RUNTIME_OPTIONS_INITIALIZER { \
+		/* general repmgr options */	\
+		"", false, false, false,		\
+		/* options which override setting in repmgr.conf */ \
+		"", "",                         \
+		/* connection parameters */		\
+		"", "", "", "", "", "", "", 	\
+		false, false, false,		    \
+		/* standby clone parameters */  \
+		false, DEFAULT_WAL_KEEP_SEGMENTS, false, false, false, false, false, false, \
+		CONFIG_FILE_SAMEPATH, false, 0, "", \
+		/* witness create parameters */ \
+		false,                          \
+		/* standby follow parameters */ \
+		false,                          \
+		/* cluster {show|matrix|crosscheck} parameters */ \
+		false,                          \
+		/* cluster cleanup parameters */ \
+		0,                              \
+		/* standby switchover parameters */ \
+		"", false, "", "fast",          \
+		/* standby {archive_config | restore_config} parameters  */ \
+		"",                             \
+		/* {standby|witness} unregister parameters */ \
+		UNKNOWN_NODE_ID }
+
+struct BackupLabel
+{
+	XLogRecPtr start_wal_location;
+	char start_wal_file[MAXLEN];
+	XLogRecPtr checkpoint_location;
+	char backup_from[MAXLEN];
+	char backup_method[MAXLEN];
+	char start_time[MAXLEN];
+	char label[MAXLEN];
+	XLogRecPtr min_failover_slot_lsn;
+};
+
+
+typedef struct
+{
+	char		slot[MAXLEN];
+	char		xlog_method[MAXLEN];
+} t_basebackup_options;
+
+#define T_BASEBACKUP_OPTIONS_INITIALIZER { "", "" }
+
+typedef struct
+{
+	int    size;
+	char **keywords;
+	char **values;
+} t_conninfo_param_list;
+
+typedef struct
+{
+	char filepath[MAXPGPATH];
+	char filename[MAXPGPATH];
+	bool in_data_directory;
+} t_configfile_info;
+
+
+typedef struct
+{
+	int    size;
+	int    entries;
+	t_configfile_info **files;
+} t_configfile_list;
+
+#define T_CONFIGFILE_LIST_INITIALIZER { 0, 0, NULL }
+
+
+typedef struct
+{
+	int node_id;
+	int node_status;
+} t_node_status_rec;
+
+typedef struct
+{
+	int node_id;
+	char node_name[MAXLEN];
+	t_node_status_rec **node_status_list;
+} t_node_matrix_rec;
+
+typedef struct
+{
+	int node_id;
+	char node_name[MAXLEN];
+	t_node_matrix_rec **matrix_list_rec;
+} t_node_status_cube;
 
-	char masterport[MAXLEN];
 
-} t_runtime_options;
 
 #endif

repmgr.sql

@@ -1,7 +1,7 @@
 /*
  * repmgr.sql
  *
- * Copyright (C) 2ndQuadrant, 2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  */
 
@@ -14,8 +14,11 @@ CREATE SCHEMA repmgr;
  */
 CREATE TABLE repl_nodes (
   id            integer primary key,
-  cluster   text        not null,       -- Name to identify the cluster
-  conninfo      text    not null
+  cluster   	text    not null,       -- Name to identify the cluster
+  name			text	not null,
+  conninfo      text    not null,
+  priority  	integer not null,
+  witness   	boolean not null default false
 );
 ALTER TABLE repl_nodes OWNER TO repmgr;
 
@@ -28,13 +31,12 @@ CREATE TABLE repl_monitor (
   standby_node                   INTEGER NOT NULL,
   last_monitor_time                      TIMESTAMP WITH TIME ZONE NOT NULL,
   last_wal_primary_location      TEXT NOT NULL,
-  last_wal_standby_location      TEXT NOT NULL,
+  last_wal_standby_location      TEXT,		-- In case of a witness server this will be NULL
   replication_lag                BIGINT NOT NULL,
   apply_lag                      BIGINT NOT NULL
 );
 ALTER TABLE repl_monitor OWNER TO repmgr;
 
-
 /*
  * This view shows the latest monitor info about every node.
  * Interesting thing to see:
@@ -46,14 +48,23 @@ ALTER TABLE repl_monitor OWNER TO repmgr;
  * time_lag: how many seconds are we from being up-to-date with master
  */
 CREATE VIEW repl_status AS
-WITH monitor_info AS (SELECT *, ROW_NUMBER() OVER (PARTITION BY primary_node, standby_node
-                                                       ORDER BY last_monitor_time desc)
-                        FROM repl_monitor)
-SELECT primary_node, standby_node, last_monitor_time, last_wal_primary_location,
+SELECT primary_node, standby_node, name AS standby_name, last_monitor_time, last_wal_primary_location,
        last_wal_standby_location, pg_size_pretty(replication_lag) replication_lag,
        pg_size_pretty(apply_lag) apply_lag,
        age(now(), last_monitor_time) AS time_lag
-  FROM monitor_info a
- WHERE row_number = 1;
+ FROM repl_monitor JOIN repl_nodes ON standby_node = id
+WHERE (standby_node, last_monitor_time) IN (SELECT standby_node, MAX(last_monitor_time)
+                                              FROM repl_monitor GROUP BY 1);
 
 ALTER VIEW repl_status OWNER TO repmgr;
+
+CREATE INDEX idx_repl_status_sort ON repl_monitor(last_monitor_time, standby_node);
+
+/*
+ * This view shows the list of nodes with the information of which one is the upstream
+ * in each case (when appliable)
+ */
+CREATE VIEW repl_show_nodes AS
+SELECT rn.id, rn.conninfo, rn.type, rn.name, rn.cluster,
+	rn.priority, rn.active, sq.name AS upstream_node_name
+FROM repl_nodes as rn LEFT JOIN repl_nodes AS sq ON sq.id=rn.upstream_node_id;

sql/Makefile

@@ -0,0 +1,21 @@
+#
+# Makefile
+#
+# Copyright (c) 2ndQuadrant, 2010-2016
+#
+
+MODULE_big = repmgr_funcs
+DATA_built=repmgr_funcs.sql
+DATA=uninstall_repmgr_funcs.sql
+OBJS=repmgr_funcs.o
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/repmgr/sql
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif

sql/repmgr2_repmgr3.sql

@@ -0,0 +1,85 @@
+/*
+ * Update a repmgr 2.x installation to repmgr 3.0
+ * ----------------------------------------------
+ *
+ * 1. Stop any running repmgrd instances
+ * 2. On the master node, execute the SQL statements listed below,
+ *    taking care to identify the master node and any inactive
+ *    nodes
+ * 3. Restart repmgrd (being sure to use repmgr 3.0)
+ */
+
+/*
+ * Set the search path to the name of the schema used by
+ * your repmgr installation
+ * (this should be "repmgr_" + the cluster name defined in
+ * 'repmgr.conf')
+ */
+
+-- SET search_path TO 'name_of_repmgr_schema';
+
+BEGIN;
+
+ALTER TABLE repl_nodes RENAME TO repl_nodes2_0;
+
+CREATE TABLE repl_nodes (
+  id               INTEGER PRIMARY KEY,
+  type             TEXT    NOT NULL CHECK (type IN('master','standby','witness')),
+  upstream_node_id INTEGER NULL REFERENCES repl_nodes (id),
+  cluster          TEXT    NOT NULL,
+  name             TEXT    NOT NULL,
+  conninfo         TEXT    NOT NULL,
+  slot_name        TEXT    NULL,
+  priority         INTEGER NOT NULL,
+  active           BOOLEAN NOT NULL DEFAULT TRUE
+);
+
+INSERT INTO repl_nodes
+           (id, type, cluster, name, conninfo, priority)
+     SELECT id,
+            CASE
+               WHEN witness IS TRUE THEN 'witness'
+               ELSE 'standby'
+            END AS type,
+            cluster,
+            name,
+            conninfo,
+            priority + 100
+       FROM repl_nodes2_0;
+
+/*
+ * You'll need to set the master explicitly; the following query
+ * should identify the master node ID but will only work if all
+ * standby servers are connected:
+ *
+ * SELECT id FROM repmgr_test.repl_nodes WHERE name NOT IN (SELECT application_name FROM pg_stat_replication)
+ *
+ * If in doubt, execute 'repmgr cluster show' will definitively identify
+ * the master.
+ */
+UPDATE repl_nodes SET type = 'master' WHERE id = $master_id;
+
+/* If any nodes are known to be inactive, update them here */
+
+-- UPDATE repl_nodes SET active = FALSE WHERE id IN (...);
+
+/* There's also an event table which we need to create */
+CREATE TABLE repl_events (
+  node_id          INTEGER NOT NULL,
+  event            TEXT NOT NULL,
+  successful       BOOLEAN NOT NULL DEFAULT TRUE,
+  event_timestamp  TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
+  details          TEXT NULL
+);
+
+/* When you're sure of your changes, commit them */
+
+-- COMMIT;
+
+
+/*
+ * execute the following command when you are sure you no longer
+ * require the old table:
+ */
+
+-- DROP TABLE repl_nodes2_0;

sql/repmgr3.0_repmgr3.1.sql

@@ -0,0 +1,35 @@
+/*
+ * Update a repmgr 3.0 installation to repmgr 3.1
+ * ----------------------------------------------
+ *
+ * The new repmgr package should be installed first. Then
+ * carry out these steps:
+ *
+ *   1. (If repmgrd is used) stop any running repmgrd instances
+ *   2. On the master node, execute the SQL statements listed below
+ *   3. (If repmgrd is used) restart repmgrd
+ */
+
+/*
+ * If your repmgr installation is not included in your repmgr
+ * user's search path, please set the search path to the name
+ * of the repmgr schema to ensure objects are installed in
+ * the correct location.
+ *
+ * The repmgr schema is  "repmgr_" + the cluster name defined in
+ * 'repmgr.conf'.
+ */
+
+-- SET search_path TO 'name_of_repmgr_schema';
+
+BEGIN;
+
+-- New view "repl_show_nodes" which also displays the server's
+-- upstream node
+
+CREATE VIEW repl_show_nodes AS
+SELECT rn.id, rn.conninfo, rn.type, rn.name, rn.cluster,
+	rn.priority, rn.active, sq.name AS upstream_node_name
+FROM repl_nodes as rn LEFT JOIN repl_nodes AS sq ON sq.id=rn.upstream_node_id;
+
+COMMIT;

sql/repmgr3.1.1_repmgr3.1.2.sql

@@ -0,0 +1,32 @@
+/*
+ * Update a repmgr 3.1.1 installation to repmgr 3.1.2
+ * --------------------------------------------------
+ *
+ * This update is only required if repmgrd is being used in conjunction
+ * with a witness server.
+ *
+ * The new repmgr package should be installed first. Then
+ * carry out these steps:
+ *
+ *   1. (If repmgrd is used) stop any running repmgrd instances
+ *   2. On the master node, execute the SQL statement listed below
+ *   3. (If repmgrd is used) restart repmgrd
+ */
+
+/*
+ * If your repmgr installation is not included in your repmgr
+ * user's search path, please set the search path to the name
+ * of the repmgr schema to ensure objects are installed in
+ * the correct location.
+ *
+ * The repmgr schema is  "repmgr_" + the cluster name defined in
+ * 'repmgr.conf'.
+ */
+
+-- SET search_path TO 'name_of_repmgr_schema';
+
+BEGIN;
+
+ALTER TABLE repl_nodes DROP CONSTRAINT repl_nodes_upstream_node_id_fkey,
+      ADD CONSTRAINT repl_nodes_upstream_node_id_fkey FOREIGN KEY (upstream_node_id) REFERENCES repl_nodes(id) DEFERRABLE;
+COMMIT;

sql/repmgr_funcs.c

@@ -0,0 +1,245 @@
+/*
+ * repmgr_funcs.c
+ * Copyright (c) 2ndQuadrant, 2010
+ *
+ * Shared memory state management and some backend functions in SQL
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "replication/walreceiver.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "storage/procarray.h"
+#include "storage/shmem.h"
+#include "storage/spin.h"
+#include "utils/builtins.h"
+#include "utils/timestamp.h"
+
+/* same definition as the one in xlog_internal.h */
+#define MAXFNAMELEN		64
+
+PG_MODULE_MAGIC;
+
+/*
+ * Global shared state
+ */
+typedef struct repmgrSharedState
+{
+	LWLockId	lock;			/* protects search/modification */
+	char		location[MAXFNAMELEN];	/* last known xlog location */
+	TimestampTz last_updated;
+}	repmgrSharedState;
+
+/* Links to shared memory state */
+static repmgrSharedState *shared_state = NULL;
+
+static shmem_startup_hook_type prev_shmem_startup_hook = NULL;
+
+void		_PG_init(void);
+void		_PG_fini(void);
+
+static void repmgr_shmem_startup(void);
+static Size repmgr_memsize(void);
+
+static bool repmgr_set_standby_location(char *locationstr);
+
+Datum		repmgr_update_standby_location(PG_FUNCTION_ARGS);
+Datum		repmgr_get_last_standby_location(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(repmgr_update_standby_location);
+PG_FUNCTION_INFO_V1(repmgr_get_last_standby_location);
+
+Datum		repmgr_update_last_updated(PG_FUNCTION_ARGS);
+Datum		repmgr_get_last_updated(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(repmgr_update_last_updated);
+PG_FUNCTION_INFO_V1(repmgr_get_last_updated);
+
+
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+	/*
+	 * In order to create our shared memory area, we have to be loaded via
+	 * shared_preload_libraries.  If not, fall out without hooking into any of
+	 * the main system.  (We don't throw error here because it seems useful to
+	 * allow the repmgr functions to be created even when the module isn't
+	 * active.	The functions must protect themselves against being called
+	 * then, however.)
+	 */
+	if (!process_shared_preload_libraries_in_progress)
+		return;
+
+	/*
+	 * Request additional shared resources.  (These are no-ops if we're not in
+	 * the postmaster process.)  We'll allocate or attach to the shared
+	 * resources in repmgr_shmem_startup().
+	 */
+	RequestAddinShmemSpace(repmgr_memsize());
+
+#if (PG_VERSION_NUM >= 90600)
+	RequestNamedLWLockTranche("repmgr", 1);
+#else
+	RequestAddinLWLocks(1);
+#endif
+
+	/*
+	 * Install hooks.
+	 */
+	prev_shmem_startup_hook = shmem_startup_hook;
+	shmem_startup_hook = repmgr_shmem_startup;
+}
+
+/*
+ * Module unload callback
+ */
+void
+_PG_fini(void)
+{
+	/* Uninstall hooks. */
+	shmem_startup_hook = prev_shmem_startup_hook;
+}
+
+/*
+ * shmem_startup hook: allocate or attach to shared memory,
+ */
+static void
+repmgr_shmem_startup(void)
+{
+	bool		found;
+
+	if (prev_shmem_startup_hook)
+		prev_shmem_startup_hook();
+
+	/* reset in case this is a restart within the postmaster */
+	shared_state = NULL;
+
+	/*
+	 * Create or attach to the shared memory state, including hash table
+	 */
+	LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
+
+	shared_state = ShmemInitStruct("repmgr shared state",
+								   sizeof(repmgrSharedState),
+								   &found);
+
+	if (!found)
+	{
+		/* First time through ... */
+#if (PG_VERSION_NUM >= 90600)
+		shared_state->lock = &(GetNamedLWLockTranche("repmgr"))->lock;
+#else
+		shared_state->lock = LWLockAssign();
+#endif
+		snprintf(shared_state->location,
+				 sizeof(shared_state->location), "%X/%X", 0, 0);
+	}
+
+	LWLockRelease(AddinShmemInitLock);
+}
+
+
+/*
+ * Estimate shared memory space needed.
+ */
+static Size
+repmgr_memsize(void)
+{
+	return MAXALIGN(sizeof(repmgrSharedState));
+}
+
+
+static bool
+repmgr_set_standby_location(char *locationstr)
+{
+	/* Safety check... */
+	if (!shared_state)
+		return false;
+
+	LWLockAcquire(shared_state->lock, LW_EXCLUSIVE);
+	strncpy(shared_state->location, locationstr, MAXFNAMELEN);
+	LWLockRelease(shared_state->lock);
+
+	return true;
+}
+
+
+/* SQL Functions */
+
+/* Read last xlog location reported by this standby from shared memory */
+Datum
+repmgr_get_last_standby_location(PG_FUNCTION_ARGS)
+{
+	char		location[MAXFNAMELEN];
+
+	/* Safety check... */
+	if (!shared_state)
+		PG_RETURN_NULL();
+
+	LWLockAcquire(shared_state->lock, LW_SHARED);
+	strncpy(location, shared_state->location, MAXFNAMELEN);
+	LWLockRelease(shared_state->lock);
+
+	PG_RETURN_TEXT_P(cstring_to_text(location));
+}
+
+
+/* Set update last xlog location reported by this standby to shared memory */
+Datum
+repmgr_update_standby_location(PG_FUNCTION_ARGS)
+{
+	text	   *location = PG_GETARG_TEXT_P(0);
+	char	   *locationstr;
+
+	/* Safety check... */
+	if (!shared_state)
+		PG_RETURN_BOOL(false);
+
+	locationstr = text_to_cstring(location);
+
+	PG_RETURN_BOOL(repmgr_set_standby_location(locationstr));
+}
+
+/* update and return last updated with current timestamp */
+Datum
+repmgr_update_last_updated(PG_FUNCTION_ARGS)
+{
+	TimestampTz last_updated = GetCurrentTimestamp();
+
+	/* Safety check... */
+	if (!shared_state)
+		PG_RETURN_NULL();
+
+	LWLockAcquire(shared_state->lock, LW_SHARED);
+	shared_state->last_updated = last_updated;
+	LWLockRelease(shared_state->lock);
+
+	PG_RETURN_TIMESTAMPTZ(last_updated);
+}
+
+
+/* get last updated timestamp */
+Datum
+repmgr_get_last_updated(PG_FUNCTION_ARGS)
+{
+	TimestampTz last_updated;
+
+	/* Safety check... */
+	if (!shared_state)
+		PG_RETURN_NULL();
+
+	LWLockAcquire(shared_state->lock, LW_EXCLUSIVE);
+	last_updated = shared_state->last_updated;
+	LWLockRelease(shared_state->lock);
+
+	PG_RETURN_TIMESTAMPTZ(last_updated);
+}
+
+

sql/repmgr_funcs.sql.in

@@ -0,0 +1,23 @@
+/*
+ * repmgr_function.sql
+ * Copyright (c) 2ndQuadrant, 2010-2016
+ *
+ */
+
+-- SET SEARCH_PATH TO 'repmgr';
+
+CREATE FUNCTION repmgr_update_standby_location(text) RETURNS boolean
+AS 'MODULE_PATHNAME', 'repmgr_update_standby_location'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION repmgr_get_last_standby_location() RETURNS text
+AS 'MODULE_PATHNAME', 'repmgr_get_last_standby_location'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION repmgr_update_last_updated() RETURNS TIMESTAMP WITH TIME ZONE
+AS 'MODULE_PATHNAME', 'repmgr_update_last_updated'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION repmgr_get_last_updated() RETURNS TIMESTAMP WITH TIME ZONE
+AS 'MODULE_PATHNAME', 'repmgr_get_last_updated'
+LANGUAGE C STRICT;

sql/uninstall_repmgr_funcs.sql

@@ -0,0 +1,11 @@
+/*
+ * uninstall_repmgr_funcs.sql
+ * Copyright (c) 2ndQuadrant, 2010-2016
+ *
+ */
+
+DROP FUNCTION repmgr_update_standby_location(text);
+DROP FUNCTION repmgr_get_last_standby_location();
+
+DROP FUNCTION repmgr_update_last_updated();
+DROP FUNCTION repmgr_get_last_updated();

strutil.c

@@ -1,7 +1,7 @@
 /*
  * strutil.c
  *
- * Copyright (C) 2ndQuadrant, 2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -25,20 +25,21 @@
 #include "log.h"
 #include "strutil.h"
 
-static int xvsnprintf(char *str, size_t size, const char *format, va_list ap);
-
+static int
+xvsnprintf(char *str, size_t size, const char *format, va_list ap)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 0)));
 
 static int
 xvsnprintf(char *str, size_t size, const char *format, va_list ap)
 {
-	int retval;
+	int			retval;
 
 	retval = vsnprintf(str, size, format, ap);
 
-	if (retval >= size)
+	if (retval >= (int) size)
 	{
 		log_err(_("Buffer of size not large enough to format entire string '%s'\n"),
-		        str);
+				str);
 		exit(ERR_STR_OVERFLOW);
 	}
 
@@ -47,10 +48,10 @@ xvsnprintf(char *str, size_t size, const char *format, va_list ap)
 
 
 int
-xsnprintf(char *str, size_t size, const char *format, ...)
+xsnprintf(char *str, size_t size, const char *format,...)
 {
-	va_list arglist;
-	int retval;
+	va_list		arglist;
+	int			retval;
 
 	va_start(arglist, format);
 	retval = xvsnprintf(str, size, format, arglist);
@@ -61,7 +62,7 @@ xsnprintf(char *str, size_t size, const char *format, ...)
 
 
 int
-sqlquery_snprintf(char *str, const char *format, ...)
+sqlquery_snprintf(char *str, const char *format,...)
 {
 	va_list		arglist;
 	int			retval;
@@ -74,7 +75,8 @@ sqlquery_snprintf(char *str, const char *format, ...)
 }
 
 
-int maxlen_snprintf(char *str, const char *format, ...)
+int
+maxlen_snprintf(char *str, const char *format,...)
 {
 	va_list		arglist;
 	int			retval;
@@ -85,3 +87,34 @@ int maxlen_snprintf(char *str, const char *format, ...)
 
 	return retval;
 }
+
+
+/*
+ * Adapted from: src/fe_utils/string_utils.c
+ *
+ * Function not publicly available before PostgreSQL 9.6.
+ */
+void
+appendShellString(PQExpBuffer buf, const char *str)
+{
+	const char *p;
+
+	appendPQExpBufferChar(buf, '\'');
+	for (p = str; *p; p++)
+	{
+		if (*p == '\n' || *p == '\r')
+		{
+			fprintf(stderr,
+					_("shell command argument contains a newline or carriage return: \"%s\"\n"),
+					str);
+			exit(ERR_BAD_CONFIG);
+		}
+
+		if (*p == '\'')
+			appendPQExpBufferStr(buf, "'\"'\"'");
+		else
+			appendPQExpBufferChar(buf, *p);
+	}
+
+	appendPQExpBufferChar(buf, '\'');
+}

strutil.h

@@ -1,6 +1,6 @@
 /*
  * strutil.h
- * Copyright (C) 2ndQuadrant, 2010-2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  *
  * This program is free software: you can redistribute it and/or modify
@@ -22,7 +22,9 @@
 #define _STRUTIL_H_
 
 #include <stdlib.h>
-#include <errcode.h>
+#include "pqexpbuffer.h"
+#include "errcode.h"
+
 
 #define QUERY_STR_LEN	8192
 #define MAXLEN			1024
@@ -30,9 +32,23 @@
 #define MAXVERSIONSTR	16
 #define MAXCONNINFO		1024
 
+/* Why? http://stackoverflow.com/a/5459929/398670 */
+#define STR(x) CppAsString(x)
 
-extern int xsnprintf(char *str, size_t size, const char *format, ...);
-extern int sqlquery_snprintf(char *str, const char *format, ...);
-extern int maxlen_snprintf(char *str, const char *format, ...);
+#define MAXLEN_STR STR(MAXLEN)
 
-#endif	/* _STRUTIL_H_ */
+extern int
+xsnprintf(char *str, size_t size, const char *format,...)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
+
+extern int
+sqlquery_snprintf(char *str, const char *format,...)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
+
+extern int
+maxlen_snprintf(char *str, const char *format,...)
+__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
+
+extern void
+appendShellString(PQExpBuffer buf, const char *str);
+#endif   /* _STRUTIL_H_ */

uninstall_repmgr.sql

@@ -1,7 +1,7 @@
 /*
  * uninstall_repmgr.sql
  *
- * Copyright (C) 2ndQuadrant, 2010-2011
+ * Copyright (C) 2ndQuadrant, 2010-2016
  *
  */
 

version.h

@@ -0,0 +1,6 @@
+#ifndef _VERSION_H_
+#define _VERSION_H_
+
+#define REPMGR_VERSION "3.2.1"
+
+#endif