2nd management server won't stay connected

[Drew Miller]

I've inherited a mysql cluster. It has 2 management servers, and the 2nd of them won't connect.

I'd like to know the reason why the other servers do not want to stay connected to the 2nd management server, since usually they do.

My logs on mgm node 23 show it connects, and disconnects:

017-11-08 19:34:07 [MgmtSrvr] INFO -- Id: 23, Command port: 192.168.207.73:1186
2017-11-08 19:34:07 [MgmtSrvr] INFO -- MySQL Cluster Management Server mysql-5.6.22 ndb-7.3.8 started
2017-11-08 19:34:07 [MgmtSrvr] INFO -- Node 23: Node 23 Connected
2017-11-08 19:34:08 [MgmtSrvr] INFO -- Node 23 connected
2017-11-08 19:34:08 [MgmtSrvr] INFO -- Node 23: Node 3 Connected
2017-11-08 22:05:21 [MgmtSrvr] WARNING -- Node 3: Node 1 missed heartbeat 2
2017-11-08 22:05:36 [MgmtSrvr] WARNING -- Node 3: Node 1 missed heartbeat 3
2017-11-08 22:05:39 [MgmtSrvr] INFO -- Node 3: Communication to Node 1 closed
2017-11-08 22:05:39 [MgmtSrvr] INFO -- Node 3: Lost arbitrator node 1 - process failure [state=6]
2017-11-08 22:05:39 [MgmtSrvr] ALERT -- Node 3: Node 1 Disconnected
2017-11-08 22:05:39 [MgmtSrvr] INFO -- Node 3: Communication to Node 1 opened
2017-11-08 22:06:51 [MgmtSrvr] INFO -- Node 3: Node 1 Connected
2017-11-08 22:06:51 [MgmtSrvr] INFO -- Node 3: Node 1: API mysql-5.6.22 ndb-7.3.8
2017-11-08 22:07:00 [MgmtSrvr] INFO -- Node 3: Prepare arbitrator node 1 [ticket=e49f0005e8313d8c]

The cluster membership table looks like:

mysql> select node_id, group_id, connected_rank1_arbs, connected_rank2_arbs fbership;
+---------+----------+----------------------+----------------------+
| node_id | group_id | connected_rank1_arbs | connected_rank2_arbs |
+---------+----------+----------------------+----------------------+
| 2 | 0 | 1 | - |
| 3 | 0 | 1, 23 | - |
| 4 | 1 | 1 | - |
| 5 | 1 | 1 | - |
| 6 | 2 | 1 | - |
| 7 | 2 | 1 | - |
+---------+----------+----------------------+----------------------+

Where should I look next to dig a little deeper?

Thanks,
Drew