Rebooting arbitrator management node causes alternate management node to fail

[Brian Hobson]

cluster version 7.3
4 x RedHat Linux 5.8 x86_64 servers
32 GB system memory each
4 quad-core processors each

server1:
1 x ndb data node
8 x mysql nodes

server2:
1 x ndb data node
8 x mysql nodes

management1:
1 x ndb_mgmd node (listed as arbitrator)

management2:
1 x ndb_mgmd node

When the cluster is fully up and all nodes have joined, I restart the server where the arbitrator management node resides, management1 in the above example. I see in the logs that the management2 node has taken over as the arbitrator as expected. The odd behavior occurs when the management1 server finishes booting and the node rejoins the cluster, I see that the 'new' arbitrator management node (management2) fails with reportError messages in the log file and I see in the data node logs the following error:

Lost arbitrator node 1 - process failure [state=6]
President restarts arbitration thread [state=1]
Could not find an arbitrator, cluster is not partition-safe

When this happens, both management nodes appear active when i run 'ndb_mgm -e show' from management1 even though management2 does not appear responsive.

This behavior is repeatable in my configuration. If I manually shut down the management process before I reboot the server I cannot repeat the issue. This seems strange to me since the init.d scripts take care of shuttong down the process before rebooting. Has anyone ever experienced anything of this nature? I don't remember ever seeing this problem in ndb version 7.2.

Thanks in advance for any advice. I would be happy to provide any additional data.

Brian