repmgrd: ensure witness node marked active

Previously, if the witness node's PostgreSQL was unreachable, it would
be marked as "inactive" on the primary, and under some circumstances
would not be corrected to "active" once the witness node's PostgreSQL
came back.

PR #754; some modifications by Ian Barwick.

HISTORY

@@ -1,6 +1,7 @@
 5.3.2   2022-??-??
         standby clone: don't error out if unable to determine cluster size (Ian)
         node check: fix --downstream --nagios output; GitHub #749 (Ian)
+        repmgrd: ensure witness node marked active (hslightdb)
 
 5.3.1   2022-02-15
         repmgrd: fixes for potential connection leaks (hslightdb)

doc/appendix-release-notes.xml

@@ -46,6 +46,13 @@
               for all databases in the cluster, ignore the cluster size query if it fails.
             </para>
           </listitem>
+          <listitem>
+            <para>
+              &repmgrd;: ensure the witness node record on the primary is always marked
+              as <literal>active</literal> if previously marked <literal>inactive</literal>.
+              GitHub #754.
+            </para>
+          </listitem>
         </itemizedlist>
       </para>
     </sect2>

repmgrd-physical.c

@@ -2382,6 +2382,18 @@ monitor_streaming_witness(void)
 			log_warning(_("unable to retrieve node record from primary"));
 		}
 
+		/* refresh local node record from the primary */
+		record_status = get_node_record(primary_conn, config_file_options.node_id, &local_node_info);
+
+		if (record_status != RECORD_FOUND)
+		{
+			log_error(_("no metadata record found for this node on the current primary - terminating"));
+			log_hint(_("check that 'repmgr witness register' was executed for this node"));
+			close_connection(&primary_conn);
+			close_connection(&local_conn);
+			terminate(ERR_BAD_CONFIG);
+		}
+
 		initPQExpBuffer(&event_details);
 
 		appendPQExpBuffer(&event_details,