ndbd log message: CM_REGREF

[Paul Medynski]

My ndbd cluster nodes are logging the following at startup:

2006-04-09 15:21:12 [ndbd] INFO -- Angel pid: 15317 ndb pid: 15318
2006-04-09 15:21:12 [ndbd] INFO -- NDB Cluster -- DB node 2
2006-04-09 15:21:12 [ndbd] INFO -- Configuration fetched at node1 port 1186
2006-04-09 15:21:12 [ndbd] INFO -- WatchDog timer is set to 6000 ms
2006-04-09 15:21:12 [ndbd] INFO -- Start initiated (version 5.1.7)
2006-04-09 15:21:13 [ndbd] INFO -- Start phase 0 completed
2006-04-09 15:21:13 [ndbd] INFO -- CM_REGREF from Node 2 to our Node 2. Cause = Election without selecting new candidate

They then hang at this point. ndb_mgm reports them as:

ndb_mgm> 2 status
Node 2: starting (Phase 1) (Version 5.1.7)

and in the ndb_mgmd log I see:

2006-04-09 15:19:54 [MgmSrvr] INFO -- NDB Cluster Management Server. Version 5.1.7 (beta)
2006-04-09 15:19:54 [MgmSrvr] INFO -- Id: 1, Command port: 1186
2006-04-09 15:21:12 [MgmSrvr] INFO -- Mgmt server state: nodeid 2 reserved for ip 10.0.0.3, m_reserved_nodes 0000000
000000006.
2006-04-09 15:21:12 [MgmSrvr] INFO -- Node 1: Node 2 Connected
2006-04-09 15:21:13 [MgmSrvr] INFO -- Mgmt server state: nodeid 2 freed, m_reserved_nodes 0000000000000002.
2006-04-09 15:21:45 [MgmSrvr] INFO -- Node 2: Start phase 1 completed

I've searched around and can't find any usefel explanation of the CM_REGREF message. In fact, I've only been able to find CM_REGREF mentioned in the source code (EventLogger.hpp) which doesn't help.

I've got two ndbd cluster nodes (node ids 2 and 3), and they are both starting up with an empty fs and the --initial option. They both report the CM_REGREF log message, and neither ever finishes startup phase 1.

The useful parts of my ndb_mgmd config are:

[ndb_mgmd]
id = 1
arbitrationRank = 1
arbitrationDelay = 0
hostname = node1

[ndbd default]
noOfReplicas = 1

[ndbd]
id = 2
hostname = node2

[ndbd]
id = 3
hostname = node3

[mysqld]
id = 4
hostname = node2

[mysqld]
id = 5
hostname = node3

I experience the same behaviour whether I startup the mysqld nodes or not, but just for completeness their configs are:

[mysqld]
ndbcluster
ndb-connectstring = node1
ndb-nodeid = 4
ndb-extra-logging

[mysqld]
ndbcluster
ndb-connectstring = node1
ndb-nodeid = 5
ndb-extra-logging

Any suggestions?

-Paul