Bump version

3.1.1

HISTORY

@@ -1,4 +1,8 @@
-3.1.0   2016-01-
+3.1.1   2016-02-
+        Add '-P/--pwprompt' option for "repmgr create witness" (Ian)
+        Prevent repmgr/repmgrd running as root (Ian)
+
+3.1.0   2016-02-01
         Add "repmgr standby switchover" command (Ian)
         Revised README file (Ian)
         Remove requirement for 'archive_mode' to be enabled (Ian)

README.md

@@ -33,10 +33,14 @@ provides a single read/write master server and one or more read-only standbys
 containing near-real time copies of the master server's database.
 
 For a multi-master replication solution, please see 2ndQuadrant's BDR
-(bi-directional replication) extension. For selective replication, e.g.
-of individual tables or databases from one server to another, please
-see 2ndQuadrant's pglogical extension.
+(bi-directional replication) extension.
 
+http://2ndquadrant.com/en-us/resources/bdr/
+
+For selective replication, e.g. of individual tables or databases from one server
+to another, please see 2ndQuadrant's pglogical extension.
+
+http://2ndquadrant.com/en-us/resources/pglogical/
 
 ### Concepts
 
@@ -109,16 +113,16 @@ tables:
   - `repl_monitor`: historical standby monitoring information written by `repmgrd`
 
 views:
-  - `repl_show_nodes`: based on the `repl_nodes` showing name of the server's
-    upstream node
+  - `repl_show_nodes`: based on the table `repl_nodes`, additionally showing the
+     name of the server's  upstream node
   - `repl_status`: when `repmgrd`'s monitoring is enabled, shows current monitoring
     status for each node
 
 The `repmgr` metadata schema can be stored in an existing database or in its own
 dedicated database.
 
-A dedicated superuser is required to own the meta-database as well as carry out
-administrative actions.
+A dedicated database superuser is required to own the meta-database as well as carry
+out administrative actions.
 
 Installation
 ------------
@@ -128,7 +132,9 @@ Installation
 `repmgr` is developed and tested on Linux and OS X, but should work on any
 UNIX-like system supported by PostgreSQL itself.
 
-`repmgr` supports PostgreSQL from version 9.3.
+Current versions of `repmgr` support PostgreSQL from version 9.3. If you are
+interested in using `repmgr` on earlier versions of PostgreSQL you can download
+version 2.1 which supports PostgreSQL from version 9.1.
 
 All servers in the replication cluster must be running the same major version of
 PostgreSQL, and we recommend that they also run the same minor version.
@@ -137,7 +143,7 @@ The `repmgr` tools must be installed on each server in the replication cluster.
 
 A dedicated system user for `repmgr` is *not* required; as many `repmgr` and
 `repmgrd` actions require direct access to the PostgreSQL data directory,
-it should executed by the `postgres` user.
+it should be executed by the `postgres` user.
 
 Additionally, we recommend installing `rsync` and enabling passwordless
 `ssh` connectivity between all servers in the replication cluster.
@@ -186,7 +192,8 @@ PostgreSQL itself.
 `repmgr` and `repmgrd` use a common configuration file, by default called
 `repmgr.conf` (although any name can be used if explicitly specified).
 At the very least, `repmgr.conf` must contain the connection parameters
-for the local `repmgr` database.
+for the local `repmgr` database; see `repmgr configuration file` below
+for more details.
 
 The configuration file will be searched for in the following locations:
 
@@ -364,11 +371,11 @@ Clone the standby with:
     [2016-01-07 17:21:28] [NOTICE] you can now start your PostgreSQL server
     [2016-01-07 17:21:28] [HINT] for example : pg_ctl -D /path/to/node2/data/ start
 
-This will clone the PostgreSQL data directory files from the master using
-PostgreSQL's pg_basebackup utility. A `recovery.conf` file containing the
-correct parameters to start streaming from the master server will be created
+This will clone the PostgreSQL data directory files from the master at repmgr_node1
+using PostgreSQL's pg_basebackup utility. A `recovery.conf` file containing the
+correct parameters to start streaming from this master server will be created
 automatically, and unless otherwise the `postgresql.conf` and `pg_hba.conf`
-files will be copied.
+files will be copied from the master.
 
 Make any adjustments to the PostgreSQL configuration files now, then start the
 standby server.
@@ -377,12 +384,12 @@ standby server.
 
 > *NOTE*: `repmgr standby clone` does not require `repmgr.conf`, however we
 > recommend providing this as `repmgr` will set the `application_name` parameter
-> in `recovery.conf` as value provided in `node_name`, making it easier to identify
-> the node in `pg_stat_replication`. It's also possible to provide some advanced
-> options for controlling the standby cloning process; see next section for
-> details.
+> in `recovery.conf` as the value provided in `node_name`, making it easier to
+> identify the node in `pg_stat_replication`. It's also possible to provide some
+> advanced options for controlling the standby cloning process; see next section
+> for details.
 
-***
+* * *
 
 ### Verify replication is functioning
 
@@ -425,20 +432,20 @@ table:
       2 | standby |                1 | test    | node2 | host=repmgr_node2 dbname=repmgr user=repmgr |           |      100 | t
     (2 rows)
 
-The standby server now has a copy of records for all servers in the replication
-cluster. Note that the relationship between master and standby is explicitly
-defined via the `upstream_node_id` value, which shows here that the standby's
-upstream server is the replication cluster master. While of limited use
-in a simple master/standby replication cluster, this information is required
+The standby server now has a copy of the records for all servers in the
+replication cluster. Note that the relationship between master and standby is
+explicitly defined via the `upstream_node_id` value, which shows here that the
+standby's upstream server is the replication cluster master. While of limited
+use in a simple master/standby replication cluster, this information is required
 to effectively manage cascading replication (see below).
 
 
 Advanced options for cloning a standby
 --------------------------------------
 
-The above section demonstrates the simplest possible way to clone
-a standby server. Depending on your situation, finer-grained control
-over the cloning process may be necessary.
+The above section demonstrates the simplest possible way to cloneb a standby
+server. Depending on your circumstances, finer-grained controlover the cloning
+process may be necessary.
 
 ### pg_basebackup options when cloning a standby
 
@@ -449,8 +456,8 @@ However this may impact performance of the server being cloned from
 so should be used with care.
 
 Further options can be passed to the `pg_basebackup` utility via
-the `pg_basebackup_options` in `repmgr.conf`. See the PostgreSQL
-documentation for more details:
+the setting `pg_basebackup_options` in `repmgr.conf`. See the PostgreSQL
+documentation for more details of available options:
   http://www.postgresql.org/docs/current/static/app-pgbasebackup.html
 
 ### Using rsync to clone a standby
@@ -474,7 +481,7 @@ fresh clone with `pg_basebackup`.
 
 By default, `repmgr` will attempt to copy the standard configuration files
 (`postgresql.conf`, `pg_hba.conf` and `pg_ident.conf`) even if they are located
-outside of the data directory (though note currently they will be copied
+outside of the data directory (though currently they will be copied
 into the standby's data directory). To prevent this happening, when executing
 `repmgr standby clone` provide the `--ignore-external-config-files` option.
 
@@ -702,16 +709,16 @@ Performing a switchover with repmgr
 A typical use-case for replication is a combination of master and standby
 server, with the standby serving as a backup which can easily be activated
 in case of a problem with the master. Such an unplanned failover would
-normally be handled by promoting the standby, after which appropriate action
-taken to restore the old master.
+normally be handled by promoting the standby, after which an appropriate
+action must be taken to restore the old master.
 
 In some cases however it's desirable to promote the standby in a planned
 way, e.g. so maintenance can be performed on the master; this kind of switchover
 is supported by the `repmgr standby switchover` command.
 
 `repmgr standby switchover` differs from other `repmgr` actions in that it
-also performs actions on another server, for which reason both passwordless
-SSH access and the path of `repmgr.conf` on that server.
+also performs actions on another server, for which reason you must provide
+both passwordless SSH access and the path of `repmgr.conf` on that server.
 
 * * *
 
@@ -1042,7 +1049,6 @@ makes sense to create a witness server in conjunction with running
 `repmgrd`; the witness server will require its own `repmgrd` instance.
 
 
-
 repmgrd and cascading replication
 ---------------------------------
 
@@ -1159,7 +1165,7 @@ configuration file is located if `-f/--config-file` is not supplied.
 ### repmgr commands
 
 The `repmgr` command line tool accepts commands for specific servers in the
-replication in the format "`server type` `action`", or for the entire
+replication in the format "`server_type` `action`", or for the entire
 replication cluster in the format "`cluster` `action`". Each command is
 described below.
 
@@ -1247,16 +1253,29 @@ which contains connection details for the local database.
     time a failover occurs.
 
     Note that it only makes sense to create a witness server if `repmgrd`
-    is in use; see section "witness server" above.
+    is in use; see section "Using a witness server" above.
+
+    This command requires a `repmgr.conf` file containing a valid conninfo
+    string for the server to be created, as well as the other minimum required
+    parameters detailed in the section `repmgr configuration file` above.
 
     By default the witness server will use port 5499 to facilitate easier setup
-    on a server running an existing node.
+    on a server running an existing node. To use a different port, supply
+    this explicitly in the `repmgr.conf` conninfo string.
+
+    This command also requires the location of the witness server's data
+    directory to be provided (`-D/--datadir`) as well as valid connection
+    parameters for the master server.
+
+    By default this command will create a superuser and a repmgr user.
+    The `repmgr` user name will be extracted from the `conninfo` string
+    in `repmgr.conf`.
 
 * `cluster show`
 
     Displays information about each active node in the replication cluster. This
     command polls each registered server and shows its role (master / standby /
-    witness) or "FAILED" if the node doesn't respond. It polls each server
+    witness) or `FAILED` if the node doesn't respond. It polls each server
     directly and can be run on any node in the cluster; this is also useful
     when analyzing connectivity from a particular node.
 

repmgr.c

@@ -175,12 +175,14 @@ main(int argc, char **argv)
 		{"terse", required_argument, NULL, 't'},
 		{"mode", required_argument, NULL, 'm'},
 		{"remote-config-file", required_argument, NULL, 'C'},
+		/* deprecated from 3.2; replaced with -P/----pwprompt */
 		{"initdb-no-pwprompt", no_argument, NULL, 1},
 		{"check-upstream-config", no_argument, NULL, 2},
 		{"recovery-min-apply-delay", required_argument, NULL, 3},
 		{"ignore-external-config-files", no_argument, NULL, 4},
 		{"config-archive-dir", required_argument, NULL, 5},
 		{"pg_rewind", optional_argument, NULL, 6},
+		{"pwprompt", optional_argument, NULL, 7},
 		{"help", no_argument, NULL, '?'},
 		{"version", no_argument, NULL, 'V'},
 		{NULL, 0, NULL, 0}
@@ -196,6 +198,19 @@ main(int argc, char **argv)
 
 	set_progname(argv[0]);
 
+	/* Disallow running as root to prevent directory ownership problems */
+	if (geteuid() == 0)
+	{
+		fprintf(stderr,
+				_("%s: cannot be run as root\n"
+				  "Please log in (using, e.g., \"su\") as the "
+				  "(unprivileged) user that owns\n"
+				  "the data directory.\n"
+				),
+				progname());
+		exit(1);
+	}
+
 	/* Initialise some defaults */
 
 	/* set default user */
@@ -210,7 +225,7 @@ main(int argc, char **argv)
 		}
 		else
 		{
-			fprintf(stderr, "could not get current user name: %s\n", strerror(errno));
+			fprintf(stderr, _("could not get current user name: %s\n"), strerror(errno));
 			exit(ERR_BAD_CONFIG);
 		}
 	}
@@ -405,6 +420,9 @@ main(int argc, char **argv)
 				}
 				pg_rewind_supplied = true;
 				break;
+			case 7:
+				runtime_options.witness_pwprompt = true;
+				break;
 
 			default:
 			{
@@ -746,6 +764,8 @@ do_cluster_show(void)
 			  "  FROM %s.repl_show_nodes",
 			  get_repmgr_schema_quoted(conn));
 
+	log_verbose(LOG_DEBUG, "do_cluster_show(): \n%s\n",sqlquery );
+
 	res = PQexec(conn, sqlquery);
 
 	if (PQresultStatus(res) != PGRES_TUPLES_OK)
@@ -3358,6 +3378,8 @@ do_witness_create(void)
 	char		master_hba_file[MAXLEN];
 	bool        success;
 	bool		record_created;
+	char		repmgr_user[MAXLEN];
+	char		repmgr_db[MAXLEN];
 
 	/* Connection parameters for master only */
 	keywords[0] = "host";
@@ -3365,6 +3387,13 @@ do_witness_create(void)
 	keywords[1] = "port";
 	values[1] = runtime_options.masterport;
 
+	/*
+	 * Extract the repmgr user and database names from the conninfo string
+	 * provided in repmgr.conf
+	 */
+	get_conninfo_value(options.conninfo, "user", repmgr_user);
+	get_conninfo_value(options.conninfo, "dbname", repmgr_db);
+
 	/* We need to connect to check configuration and copy it */
 	masterconn = establish_db_connection_by_params(keywords, values, true);
 	if (!masterconn)
@@ -3454,7 +3483,7 @@ do_witness_create(void)
 	maxlen_snprintf(script, "%s %s -D %s init -o \"%s-U %s\"",
 				   make_pg_path("pg_ctl"),
 				   options.pg_ctl_options, runtime_options.dest_dir,
-				   runtime_options.initdb_no_pwprompt ? "" : "-W ",
+				   runtime_options.witness_pwprompt ? "-W " : "",
 				   runtime_options.superuser);
 	log_info(_("initializing cluster for witness: %s.\n"), script);
 
@@ -3500,8 +3529,8 @@ do_witness_create(void)
 	xsnprintf(buf, sizeof(buf), "\n#Configuration added by %s\n", progname());
 	fputs(buf, pg_conf);
 
-
-	/* Attempt to extract a port number from the provided conninfo string
+	/*
+	 * Attempt to extract a port number from the provided conninfo string.
 	 * This will override any value provided with '-l/--local-port', as it's
 	 * what we'll later try and connect to anyway. '-l/--local-port' should
 	 * be deprecated.
@@ -3552,13 +3581,18 @@ do_witness_create(void)
 		exit(ERR_BAD_CONFIG);
 	}
 
+
 	/* check if we need to create a user */
-	if (runtime_options.username[0] && runtime_options.localport[0] && strcmp(runtime_options.username,"postgres") != 0)
+	if (strcmp(repmgr_user, "postgres") != 0)
 	{
-		/* create required user; needs to be superuser to create untrusted language function in c */
-		maxlen_snprintf(script, "%s -p %s --superuser --login -U %s %s",
+		/* create required user; needs to be superuser to create untrusted
+		 * language function in C */
+		maxlen_snprintf(script, "%s -p %s --superuser --login %s-U %s %s",
 						make_pg_path("createuser"),
-						runtime_options.localport, runtime_options.superuser, runtime_options.username);
+						runtime_options.localport,
+						runtime_options.witness_pwprompt ? "-P " : "",
+						runtime_options.superuser,
+						repmgr_user);
 		log_info(_("creating user for witness db: %s.\n"), script);
 
 		r = system(script);
@@ -3584,7 +3618,10 @@ do_witness_create(void)
 		/* create required db */
 		maxlen_snprintf(script, "%s -p %s -U %s --owner=%s %s",
 						make_pg_path("createdb"),
-						runtime_options.localport, runtime_options.superuser, runtime_options.username, runtime_options.dbname);
+						runtime_options.localport,
+						runtime_options.superuser,
+						repmgr_user,
+						repmgr_db);
 		log_info("creating database for witness db: %s.\n", script);
 
 		r = system(script);
@@ -3610,7 +3647,7 @@ do_witness_create(void)
 
 	if (success == false)
 	{
-		char *errmsg = _("unable to retrieve location of pg_hba.conf");
+		char *errmsg = _("Unable to retrieve location of pg_hba.conf");
 		log_err("%s\n", errmsg);
 
 		create_event_record(masterconn,
@@ -3627,7 +3664,7 @@ do_witness_create(void)
 						  master_hba_file, runtime_options.dest_dir, false, -1);
 	if (r != 0)
 	{
-		char *errmsg = _("unable to copy pg_hba.conf from master");
+		char *errmsg = _("Unable to copy pg_hba.conf from master");
 		log_err("%s\n", errmsg);
 
 		create_event_record(masterconn,
@@ -3641,7 +3678,7 @@ do_witness_create(void)
 		exit(ERR_BAD_CONFIG);
 	}
 
-	/* reload to adapt for changed pg_hba.conf */
+	/* reload witness server to activate the copied pg_hba.conf */
 	maxlen_snprintf(script, "%s %s -w -D %s reload",
 					make_pg_path("pg_ctl"),
 					options.pg_ctl_options, runtime_options.dest_dir);
@@ -3663,7 +3700,47 @@ do_witness_create(void)
 		exit(ERR_BAD_CONFIG);
 	}
 
-	/* register ourselves in the master */
+	/* establish a connection to the witness, and create the schema */
+	witnessconn = establish_db_connection(options.conninfo, false);
+
+	if (PQstatus(witnessconn) != CONNECTION_OK)
+	{
+		create_event_record(masterconn,
+							&options,
+							options.node,
+							"witness_create",
+							false,
+							_("Unable to connect to witness servetr"));
+		PQfinish(masterconn);
+		exit(ERR_BAD_CONFIG);
+	}
+
+
+	log_info(_("starting copy of configuration from master...\n"));
+
+	begin_transaction(witnessconn);
+
+	if (!create_schema(witnessconn))
+	{
+		rollback_transaction(witnessconn);
+		create_event_record(masterconn,
+							&options,
+							options.node,
+							"witness_create",
+							false,
+							_("Unable to create schema on witness"));
+		PQfinish(masterconn);
+		PQfinish(witnessconn);
+		exit(ERR_BAD_CONFIG);
+	}
+
+	commit_transaction(witnessconn);
+
+	/*
+	 * Register new witness server on the primary
+	 * Do this as late as possible to avoid having to delete
+	 * the record if the server creation fails
+	 */
 
 	if (runtime_options.force)
 	{
@@ -3702,29 +3779,6 @@ do_witness_create(void)
 		exit(ERR_DB_QUERY);
 	}
 
-	/* establish a connection to the witness, and create the schema */
-	witnessconn = establish_db_connection(options.conninfo, true);
-
-	log_info(_("starting copy of configuration from master...\n"));
-
-	begin_transaction(witnessconn);
-
-
-	if (!create_schema(witnessconn))
-	{
-		rollback_transaction(witnessconn);
-		create_event_record(masterconn,
-							&options,
-							options.node,
-							"witness_create",
-							false,
-							_("unable to create schema on witness"));
-		PQfinish(masterconn);
-		PQfinish(witnessconn);
-		exit(ERR_BAD_CONFIG);
-	}
-
-	commit_transaction(witnessconn);
 
 	/* copy configuration from master, only repl_nodes is needed */
 	if (!copy_configuration(masterconn, witnessconn, options.cluster_name))
@@ -3735,24 +3789,33 @@ do_witness_create(void)
 							"witness_create",
 							false,
 							_("Unable to copy configuration from master"));
+
+		/*
+		 * delete previously created witness node record
+		 * XXX maybe set inactive?
+		 */
+		delete_node_record(masterconn,
+						   options.node,
+						   "witness create");
+
 		PQfinish(masterconn);
 		PQfinish(witnessconn);
 		exit(ERR_BAD_CONFIG);
 	}
 
 	/* drop superuser powers if needed */
-	if (runtime_options.username[0] && runtime_options.localport[0] && strcmp(runtime_options.username,"postgres") != 0)
+	if (strcmp(repmgr_user, "postgres") != 0)
 	{
-		sqlquery_snprintf(sqlquery, "ALTER ROLE %s NOSUPERUSER", runtime_options.username);
+		sqlquery_snprintf(sqlquery, "ALTER ROLE %s NOSUPERUSER", repmgr_user);
 		log_info(_("revoking superuser status on user %s: %s.\n"),
-				   runtime_options.username, sqlquery);
+				   repmgr_user, sqlquery);
 
 		log_debug(_("witness create: %s\n"), sqlquery);
 		res = PQexec(witnessconn, sqlquery);
 		if (!res || PQresultStatus(res) != PGRES_COMMAND_OK)
 		{
-			log_err(_("unable to alter user privileges for user %s: %s\n"),
-					runtime_options.username,
+			log_err(_("Unable to alter user privileges for user %s: %s\n"),
+					repmgr_user,
 					PQerrorMessage(witnessconn));
 			PQfinish(masterconn);
 			PQfinish(witnessconn);
@@ -3760,6 +3823,10 @@ do_witness_create(void)
 		}
 	}
 
+	/* Finished with the witness server */
+
+	PQfinish(witnessconn);
+
 	/* Log the event */
 	create_event_record(masterconn,
 						&options,
@@ -3769,7 +3836,6 @@ do_witness_create(void)
 						NULL);
 
 	PQfinish(masterconn);
-	PQfinish(witnessconn);
 
 	log_notice(_("configuration has been successfully copied to the witness\n"));
 }
@@ -3829,7 +3895,8 @@ do_help(void)
 	printf(_("  --pg_rewind[=VALUE]                 (standby switchover) 9.3/9.4 only - use pg_rewind if available,\n" \
 			 "                                        optionally providing a path to the binary\n"));
 	printf(_("  -k, --keep-history=VALUE            (cluster cleanup) retain indicated number of days of history (default: 0)\n"));
-	printf(_("  --initdb-no-pwprompt                (witness server) no superuser password prompt during initdb\n"));
+/*	printf(_("  --initdb-no-pwprompt                (witness server) no superuser password prompt during initdb\n"));*/
+	printf(_("  -P, --pwprompt                      (witness server) prompt for password when creating users\n"));
 	printf(_("  -S, --superuser=USERNAME            (witness server) superuser username for witness database\n" \
 			 "                                        (default: postgres)\n"));
 	printf(_("\n"));
@@ -4275,6 +4342,11 @@ check_parameters_for_action(const int action)
 			config_file_required = false;
 			break;
 		case WITNESS_CREATE:
+			/* Require data directory */
+			if (strcmp(runtime_options.dest_dir, "") == 0)
+			{
+				error_list_append(&cli_errors, _("-D/--data-dir required when executing WITNESS CREATE"));
+			}
 			/* allow all parameters to be supplied */
 			break;
 		case CLUSTER_SHOW:

repmgr.h

@@ -67,7 +67,7 @@ typedef struct
 	bool		force;
 	bool		wait_for_master;
 	bool		ignore_rsync_warn;
-	bool		initdb_no_pwprompt;
+	bool		witness_pwprompt;
 	bool		rsync_only;
 	bool		fast_checkpoint;
 	bool		ignore_external_config_files;
@@ -91,11 +91,12 @@ typedef struct
 
 	char		recovery_min_apply_delay[MAXLEN];
 
-	/* deprecated command line option */
+	/* deprecated command line options */
 	char		localport[MAXLEN];
+	bool		initdb_no_pwprompt;
 }	t_runtime_options;
 
-#define T_RUNTIME_OPTIONS_INITIALIZER { "", "", "", "", "", "", "", DEFAULT_WAL_KEEP_SEGMENTS, false, false, false, false, false, false, false, false, false, "smart", "", "", "", "", "", 0, "", "", "" }
+#define T_RUNTIME_OPTIONS_INITIALIZER { "", "", "", "", "", "", "", DEFAULT_WAL_KEEP_SEGMENTS, false, false, false, false, false, false, false, false, false, "smart", "", "", "", "", "", 0, "", "", "", false }
 
 extern char		repmgr_schema[MAXLEN];
 extern bool		config_file_found;

repmgrd.c

@@ -142,6 +142,20 @@ main(int argc, char **argv)
 
 	set_progname(argv[0]);
 
+	/* Disallow running as root to prevent directory ownership problems */
+	if (geteuid() == 0)
+	{
+		fprintf(stderr,
+				_("%s: cannot be run as root\n"
+				  "Please log in (using, e.g., \"su\") as the "
+				  "(unprivileged) user that owns "
+				  "the data directory.\n"
+				),
+				progname());
+		exit(1);
+	}
+
+
 	while ((c = getopt_long(argc, argv, "?Vf:vmdp:", long_options, &optindex)) != -1)
 	{
 		switch (c)

version.h

@@ -1,6 +1,6 @@
 #ifndef _VERSION_H_
 #define _VERSION_H_
 
-#define REPMGR_VERSION "3.1"
+#define REPMGR_VERSION "3.1.1"
 
 #endif