Release notes Changes to each &repmgr; release are documented in the release notes. Please read the release notes for all versions between your current version and the version you are plan to upgrade to before performing an upgrade, as there may be version-specific upgrade steps. See also: Mar ???, 2019 &repmgr; 4.3 is a major release. On Debian-based systems, including Ubuntu, if using repmgrd please ensure that in the file /etc/init.d/repmgrd, the parameter REPMGRD_OPTS contains "--daemonize=false", e.g.: # additional options REPMGRD_OPTS="--daemonize=false" For further details, see repmgrd configuration on Debian/Ubuntu. repmgr standby follow: option --upstream-node-id can now be used to specify another standby to follow. repmgr standby follow: verify that it is actually possible to follow another node. repmgr node rejoin: verify that it is actually possible to attach the node to the current primary. New commands repmgr daemon start and repmgr daemon stop: these provide a standardized way of starting and stopping repmgrd. GitHub #528. These commands require the configuration file settings repmgrd_service_start_command and repmgrd_service_stop_command in repmgr.conf to be set. repmgr daemon status additionally displays the node priority and the interval (in seconds) since the repmgrd instance last verified its upstream node was available. Add --compact option to repmgr cluster show (GitHub #521). This makes it easier to copy the output into emails, chats etc. as a compact table. repmgr cluster show: differentiate between unreachable nodes and nodes which are running but rejecting connections. This makes it possible to see whether a node is unreachable at network level, or if it is running but rejecting connections for some reason. Add --dry-run to repmgr standby promote (GitHub #522). repmgr --version-number outputs the "raw" repmgr version number (e.g. 40300). This is intended for use by scripts etc. requiring an easily parseable representation of the &repmgr; version. repmgr node check --data-directory-config option added; this is to confirm &repmgr; is correctly configured. GitHub #523. Add check to repmgr standby switchover to ensure the data directory on the demotion candidate is configured correctly in repmgr.conf. This is to ensure that &repmgr;, when remotely executed on the demotion candidate, can correctly verify that PostgreSQL on the demotion candidate was shut down cleanly. GitHub #523. repmgrd will no longer consider nodes where repmgrd is not running as promotion candidates. Previously, if repmgrd was not running on a node, but that node qualified as the promotion candidate, it would never be promoted due to the absence of a running repmgrd. Add option connection_check_type to enable selection of the method repmgrd uses to determine whether the upstream node is available. Possible values are ping (default; uses PQping() to determine server availability), connection (attempst to make a new connection to the upstream node), and query (determines server availability by executing an SQL statement on the node via the existing connection). New configuration option failover_validation_command to allow an external mechanism to validate the failover decision made by repmgrd. New configuration option standby_disconnect_on_failover to force standbys to disconnect their WAL receivers before making a failover decision. In a failover situation, repmgrd will not attempt to promote a node if another standby has already appeared (e.g. by being promoted manually). GitHub #420. &repmgr;: when executing repmgr standby switchover, prevent escaping issues with connection URIs when executing repmgr node rejoin on the demotion candidate. GitHub #525. &repmgr;: when executing repmgr witness register, chech the node to connected is actually the primary (i.e. not the witness server). GitHub #528. &repmgr;: when executing repmgr standby clone, recheck primary/upstream connection(s) after the data copy operation is complete, as these may have gone away. &repmgr;: when executing repmgr standby switchover, avoid a potential race condition when comparing received WAL on the standby to the primary's shutdown location, as the standby's walreceiver may not have yet flushed all received WAL to disk. GitHub #518. &repmgr;: when executing repmgr standby switchover, verify the standby (promotion candidate) is currently attached to the primary (demotion candidate). GitHub #519. repmgrd: on a cascaded standby, don't fail over if failover=manual. GitHub #531. repmgr cluster show: fix display of node IDs with multiple digits. ensure repmgr primary unregister behaves correctly when executed on a witness server. GitHub #548. repmgr node check will only consider physical replication slots, as the purpose of slot checks is to warn about potential issues with streaming replication standbys which are no longer attached. Wed October 24, 2018 &repmgr; 4.2 is a major release, with the main new feature being the ability to pause repmgrd, e.g. during planned maintenance operations. Various other usability enhancements and a couple of bug fixes are also included; see notes below for details. A restart of the PostgreSQL server is required for this release. For detailed upgrade instructions, see Upgrading a major version release. On Debian-based systems, including Ubuntu, if using repmgrd please ensure that the in the file /etc/init.d/repmgrd, the parameter REPMGRD_OPTS contains "--daemonize=false", e.g.: # additional options REPMGRD_OPTS="--daemonize=false" For further details, see repmgrd daemon configuration on Debian/Ubuntu. New parameter shutdown_check_timeout (default: 60 seconds) added; this provides an explicit timeout for repmgr standby switchover to check that the demotion candidate (current primary) has shut down. Previously, the parameters reconnect_attempts and reconnect_interval were used to calculate a timeout, but these are actually intended for primary failure detection. (GitHub #504). New parameter repmgr_bindir added, to facilitate remote invocation of repmgr when the repmgr binary is located somewhere other than the PostgreSQL binary directory, as it cannot be assumed all package maintainers will install &repmgr; there. This parameter is optional; if not set (the default), &repmgr; will fall back to pg_bindir (if set). (GitHub #246). repmgr cluster cleanup now accepts the --node-id option to delete records for only one node. (GitHub #493). When running repmgr cluster matrix and repmgr cluster crosscheck, &repmgr; will report nodes unreachable via SSH, and emit return code ERR_BAD_SSH. (GitHub #246). Users relying on repmgr cluster crosscheck to return a non-zero return code as a way of detecting connectivity errors should be aware that ERR_BAD_SSH will be returned if there is an SSH connection error from the node where the command is executed, even if the command is able to establish that PostgreSQL connectivity is fine. Therefore the exact return code should be checked to determine what kind of connectivity error has been detected. repmgrd can now be "paused", i.e. instructed not to take any action such as a failover, even if the prerequisites for such an action are detected. This removes the need to stop repmgrd on all nodes when performing a planned operation such as a switchover. For further details, see Pausing repmgrd. &repmgr;: fix "Missing replication slots" label in repmgr node check. (GitHub #507) repmgrd: fix parsing of -d/--daemonize option. Wed September 5, 2018 repmgr 4.1.1 contains a number of usability enhancements and bug fixes. We recommend upgrading to this version as soon as possible. This release can be installed as a simple package upgrade from repmgr 4.0 ~ 4.1.0; repmgrd (if running) should be restarted. See for more details. repmgr standby switchover --dry-run no longer copies external configuration files to test they can be copied; this avoids making any changes to the target system. (GitHub #491). repmgr cluster cleanup: add cluster_cleanup event. (GitHub #492). repmgr standby switchover: improve detection of free walsenders. (GitHub #495). Improve messages emitted during repmgr standby promote. Always reopen the log file after receiving SIGHUP. Previously this only happened if a configuration file change was detected. (GitHub #485). Report version number after logger initialisation. (GitHub #487). Improve cascaded standby failover handling. (GitHub #480). Improve reconnection handling after brief network outages; if monitoring data being collected, this could lead to orphaned sessions on the primary. (GitHub #480). Check promote_command and follow_command are defined when reloading configuration. These were checked on startup but not reload by repmgrd, which made it possible to make repmgrd with invalid values. It's unlikely anyone would want to do this, but we should make it impossible anyway. (GitHub #486). Text of any failed queries will now be logged as ERROR to assist logfile analysis at log levels higher than DEBUG. (GitHub #498). repmgr node rejoin: remove new upstream's replication slot if it still exists on the rejoined standby. (GitHub #499). repmgrd: fix startup on witness node when local data is stale. (GitHub #488, #489). Truncate version string reported by PostgreSQL if necessary; some distributions insert additional detail after the actual version. (GitHub #490). Tue July 31, 2018 &repmgr; 4.1.0 introduces some changes to repmgrd behaviour and some additional configuration parameters. This release can be installed as a simple package upgrade from repmgr 4.0 ~ 4.0.6. The following post-upgrade steps must be carried out: Execute ALTER EXTENSION repmgr UPDATE on the primary server in the database where &repmgr; is installed. repmgrd must be restarted on all nodes where it is running. A restart of the PostgreSQL server is not required for this release (unless upgrading from repmgr 3.x). See for more details. Configuration changes are backwards-compatible and no changes to repmgr.conf are required. However users should review the changes listed below. Repository changes Coinciding with this release, the 2ndQuadrant repository structure has changed. See section for details, particularly if you are using a RPM-based system. Default for is now INFO. This produces additional informative log output, without creating excessive additional log file volume, and matches the setting assumed for examples in the documentation. (GitHub #470). recovery_min_apply_delay now accepts a minimum value of zero (GitHub #448). repmgr: always exit with an error if an unrecognised command line option is provided. This matches the behaviour of other PostgreSQL utilities such as psql. (GitHub #464). repmgr: add -q/--quiet option to suppress non-error output. (GitHub #468). repmgr cluster show, repmgr node check and repmgr node status return non-zero exit code if node status issues detected. (GitHub #456). Add --csv output option for repmgr cluster event. (GitHub #471). repmgr witness unregister can be run on any node, by providing the ID of the witness node with --node-id. (GitHub #472). repmgr standby switchover will refuse to run if an exclusive backup is taking place on the current primary. (GitHub #476). repmgrd: create a PID file by default (GitHub #457). For details, see . repmgrd: daemonize process by default. In case, for whatever reason, the user does not wish to daemonize the process, provide --daemonize=false. (GitHub #458). repmgr standby register --wait-sync: fix behaviour when no timeout provided. repmgr cluster cleanup: add missing help options. (GitHub #461/#462). Ensure witness node follows new primary after switchover. (GitHub #453). repmgr node check and repmgr node status: fix witness node handling. (GitHub #451). When using repmgr standby clone with --recovery-conf-only and replication slots, ensure primary_slot_name is set correctly. (GitHub #474). Thu June 14, 2018 &repmgr; 4.0.6 contains a number of bug fixes and usability enhancements. We recommend upgrading to this version as soon as possible. This release can be installed as a simple package upgrade from repmgr 4.0 ~ 4.0.5; repmgrd (if running) should be restarted. See for more details. repmgr cluster crosscheck and repmgr cluster matrix: return non-zero exit code if node connection issues detected (GitHub #447) repmgr standby clone: Improve handling of external configuration file copying, including consideration in --dry-run check (GitHub #443) When using --dry-run, force log level to INFO to ensure output will always be displayed (GitHub #441) repmgr standby clone: Improve documentation of --recovery-conf-only mode (GitHub #438) repmgr standby clone: Don't require presence of user parameter in conninfo string (GitHub #437) repmgr witness register: prevent registration of a witness server with the same name as an existing node repmgr standby follow: check node has actually connected to new primary before reporting success (GitHub #444) repmgr node rejoin: Fix bug when parsing --config-files parameter (GitHub #442) repmgrd: ensure local node is counted as quorum member (GitHub #439) Wed May 2, 2018 &repmgr; 4.0.5 contains a number of usability enhancements related to pg_rewind usage, recovery.conf generation and (in repmgrd) handling of various corner-case situations, as well as a number of bug fixes. Various documentation improvements, with particular emphasis on the importance of setting appropriate service commands instead of relying on pg_ctl. Poll demoted primary after restart as a standby during a switchover operation (GitHub #408). Add configuration parameter config_directory (GitHub #424). Add sanity check if --upstream-node-id not supplied when executing (GitHub #395). Enable pg_rewind to be used with PostgreSQL 9.3/9.4 (GitHub #413). When generating replication connection strings, set dbname=replication if appropriate (GitHub #421). Enable provision of archive_cleanup_command in recovery.conf (GitHub #416). Actively check for node to rejoin cluster (GitHub #415). repmgrd: set connect_timeout=2 (if not explicitly set) when pinging a server. Fix display of conninfo parsing error messages. Fix minimum accepted value for degraded_monitoring_timeout (GitHub #411). Fix superuser password handling (GitHub #400) Fix parsing of archive_ready_critical configuration file parameter (GitHub #426). Fix repmgr cluster crosscheck output (GitHub #389) Fix memory leaks in witness code (GitHub #402). repmgrd: handle pg_ctl promote timeout (GitHub #425). repmgrd: handle failover situation with only two nodes in the primary location, and at least one node in another location (GitHub #407). repmgrd: prevent standby connection handle from going stale. Fri Mar 9, 2018 &repmgr; 4.0.4 contains some bug fixes and and a number of usability enhancements related to logging/diagnostics, event notifications and pre-action checks. This release can be installed as a simple package upgrade from repmgr 4.0 ~ 4.0.3; repmgrd (if running) should be restarted. See for more details. It is not possible to perform a switchover where the demotion candidate is running &repmgr; 4.0.2 or lower; all nodes should be upgraded to the latest version (4.0.4). This is due to additional checks introduced in 4.0.3 which require the presence of 4.0.3 or later versions on all nodes. add repmgr standby clone --recovery-conf-only option to enable integration of a standby cloned from another source into a &repmgr; cluster (GitHub #382) remove restriction on using replication slots when cloning from a Barman server (GitHub #379) make repmgr standby promote timeout values configurable (GitHub #387) add missing options to main --help output (GitHub #391, #392) ensure repmgr node rejoin honours the --dry-run option (GitHub #383) improve replication slot warnings generated by repmgr node status (GitHub #385) fix --superuser handling when cloning a standby (GitHub #380) repmgrd: improve detection of status change from primary to standby repmgrd: improve reconnection to the local node after a failover (previously a connection error due to the node starting up was being interpreted as the node being unavailable) repmgrd: when running on a witness server, correctly connect to new primary after a failover repmgrd: add event notification repmgrd_shutdown (GitHub #393) Thu Feb 15, 2018 &repmgr; 4.0.3 contains some bug fixes and and a number of usability enhancements related to logging/diagnostics, event notifications and pre-action checks. This release can be installed as a simple package upgrade from repmgr 4.0 ~ 4.0.2; repmgrd (if running) should be restarted. It is not possible to perform a switchover where the demotion candidate is running &repmgr; 4.0.2 or lower; all nodes should be upgraded to 4.0.3. This is due to additional checks introduced in 4.0.3 which require the presence of 4.0.3 or later versions on all nodes. improve repmgr standby switchover behaviour when pg_ctl is used to control the server and logging output is not explicitly redirected improve repmgr standby switchover log messages and provide new exit code ERR_SWITCHOVER_INCOMPLETE when old primary could not be shut down cleanly add check to verify the demotion candidate can make a replication connection to the promotion candidate before executing a switchover (GitHub #370) add check for sufficient walsenders and replication slots on the promotion candidate before executing repmgr standby switchover (GitHub #371) add --dry-run mode to repmgr standby follow (GitHub #368) provide information about the primary node for repmgr standby register and repmgr standby follow event notifications (GitHub #375) add standby_register_sync event notification, which is fired when repmgr standby register is run with the --wait-sync option and the new or updated standby node record has synchronised to the standby (GitHub #374) when running repmgr cluster show, if any node is unreachable, output the error message encountered in the list of warnings (GitHub #369) ensure an inactive data directory can be overwritten when cloning a standby (GitHub #366) repmgr node status upstream node display fixed (GitHub #363) repmgr primary unregister: clarify usage and fix --help output (GitHub #373) parsing of pg_basebackup_options fixed (GitHub #376) ensure the pg_subtrans directory is created when cloning a standby in Barman mode repmgr witness register: fix primary node check (GitHub #377). Thu Jan 18, 2018 &repmgr; 4.0.2 contains some bug fixes and small usability enhancements. This release can be installed as a simple package upgrade from &repmgr; 4.0.1 or 4.0; repmgrd (if running) should be restarted. Recognize the -t/--terse option for repmgr cluster event to hide the Details column (GitHub #360) Add "--wait-start" option for repmgr standby register (GitHub #356) Add %p event notification parameter for repmgr standby switchover Add missing -W option to getopt_long() invocation (GitHub #350) Automatically create slot name if missing (GitHub #343) Fixes to parsing output of remote repmgr invocations (GitHub #349) When registering BDR nodes, automatically create missing connection replication set (GitHub #347) Handle missing node record in repmgr node rejoin (GitHub #358) The documentation can now be built as a single HTML file (GitHub pull request #353) Wed Dec 13, 2017 &repmgr; 4.0.1 is a bugfix release. ensure correct return codes are returned for repmgr node check --action= operations (GitHub #340) Fix when repmgr schema not set in search path (GitHub #341) When using --force-rewind with delete any replication slots copied by pg_rewind (GitHub #334) Only perform sanity check on accessibility of configuration files outside the data directory when --copy-external-config-files provided (GitHub #342) Initialise "voting_term" table in application, not extension SQL (GitHub #344) Tue Nov 21, 2017 repmgr 4.0 is an entirely new version of &repmgr;, implementing &repmgr; as a native PostgreSQL extension, adding new and improving existing features, and making &repmgr; more user-friendly and intuitive to use. The new code base will make it easier to add additional functionality for future releases. With the new version, the opportunity has been taken to make some changes in the way &repmgr; is set up and configured. In particular changes have been made to some configuration file settings consistency for and clarity. Changes are covered in detail below To standardise terminology, from this release primary is used to denote the read/write node in a streaming replication cluster. master is still accepted as an alias for &repmgr; commands (e.g. repmgr master register). For detailed instructions on upgrading from repmgr 3.x, see . improved switchover: the switchover process has been improved and streamlined, speeding up the switchover process and can also instruct other standbys to follow the new primary once the switchover has completed. See for more details. "--dry-run" option: many &repmgr; commands now provide a --dry-run option which will execute the command as far as possible without making any changes, which will enable possible issues to be identified before the intended operation is actually carried out. easier upgrades: &repmgr; is now implemented as a native PostgreSQL extension, which means future upgrades can be carried out by installing the upgraded package and issuing ALTER EXTENSION repmgr UPDATE. improved logging output: &repmgr; (and repmgrd) now provide more explicit logging output giving a better picture of what is going on. Where appropriate, DETAIL and HINT log lines provide additional detail and suggestions for resolving problems. Additionally, repmgrd now emits informational log lines at regular, configurable intervals to confirm that it's running correctly and which node(s) it's monitoring. automatic configuration file location in packages: Many operating system packages place the &repmgr; configuration files in a version-specific subdirectory, e.g. /etc/repmgr/9.6/repmgr.conf; &repmgr; now makes it easy for package maintainers to provide a patch with the actual file location, meaning repmgr.conf does not need to be provided explicitly. This is currently the case for 2ndQuadrant-provided .deb and .rpm packages. monitoring and status checks: New commands and providing information about a node's status and replication-related monitoring output. node rejoin: New commands enables a failed primary to be rejoined to a replication cluster, optionally using pg_rewind to synchronise its data, (note that pg_rewind may not be useable in some circumstances). automatic failover: improved detection of node status; promotion decision based on a consensual model, with the promoted primary explicitly informing other standbys to follow it. The repmgrd daemon will continue functioning even if the monitored PostgreSQL instance is down, and resume monitoring if it reappears. Additionally, if the instance's role has changed (typically from a primary to a standby, e.g. following reintegration of a failed primary using ) repmgrd will automatically resume monitoring it as a standby. new documentation: the existing documentation spread over multiple text files has been consolidated into DocBook format (as used by the main PostgreSQL project) and is now available online in HTML format. The DocBook files can easily be used to create versions of the documentation in other formats such as PDF. --dry-run: &repmgr; will attempt to perform the action as far as possible without making any changes to the database --upstream-node-id: use to specify the upstream node the standby will connect later stream from, when cloning and registering a standby. This replaces the configuration file parameter upstream_node. as the upstream node is set when the standby is initially cloned, but can change over the lifetime of an installation (due to failovers, switchovers etc.) so it's pointless/confusing keeping the original value around in repmgr.conf. repmgr --replication-user has been deprecated; it has been replaced by the configuration file option replication_user. The value (which defaults to the user provided in the conninfo string) will be stored in the &repmgr; metadata for use by and . --recovery-min-apply-delay is now a configuration file parameter recovery_min_apply_delay, to ensure the setting does not get lost when a standby follows a new upstream. --no-conninfo-password is deprecated; a password included in the environment variable PGPASSWORD will no longer be added to primary_conninfo by default; to force the inclusion of a password (not recommended), use the new configuration file parameter use_primary_conninfo_password. For details, ee section . repmgrd --monitoring-history is deprecated and is replaced by the configuration file option monitoring_history. This enables the setting to be changed without having to modify system service files. Required settings The following 4 parameters are mandatory in repmgr.conf: node_id node_name conninfo data_directory Renamed settings Some settings have been renamed for clarity and consistency: node is now node_id name is now node_name barman_server is now barman_host master_reponse_timeout is now async_query_timeout (to better indicate its purpose) The following configuration file parameters have been renamed for consistency with other parameters (and conform to the pattern used by PostgreSQL itself, which uses the prefix log_ for logging parameters): loglevel is now log_level logfile is now log_file logfacility is now log_facility Removed settings cluster has been removed upstream_node - see note about --upstream-node-id above retry_promote_interval_secsthis is now redundant due to changes in the failover/promotion mechanism; the new equivalent is primary_notification_timeout Logging changes default value for log_level is INFO rather than NOTICE. new parameter log_status_interval, which causes repmgrd to emit a status log line at the specified interval The shared library has been renamed from repmgr_funcs to repmgr, meaning shared_preload_libraries in postgresql.conf needs to be updated to the new name: shared_preload_libraries = 'repmgr'