missing cluster tables

[Jamie Richardson]

I’ve been testing a MySQL cluster (version 5.0.22) in our environment. I was getting an error telling me to increase MaxNoOfConcurrentOperations. I update the config.ini file on my management server. From my understanding when updating the config.ini file the cluster nodes need to be restarted with ndbd -–initial. I ran a kill -9 on the PID’s and then restarted the nodes with ndbd -–initial. I got the following error in ndb_mgm:

ndb_mgm> Node 2: Forced node shutdown completed. Occured during startphase 4. Initiated by signal 0. Caused by error 6303: 'Unable to find a restorable replica(Ndbd file system inconsistency error, please report a bug). Ndbd file system error, restart node initial
Node 3: Forced node shutdown completed. Occured during startphase 2. Initiated by signal 0. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.

I killed the node processes again and restarted them. This time everything came up fine but I cannot see any of the tables that were stored on the cluster! Did calling ndbd -–initial drop the stored tables? I could swear I’ve read that after making changes to the config.ini file nodes should be brought up with the -–initial option to grab changes from the management server. I’ve restarted the sqld that connects to the cluster and still see nothing. Are my tables gone? Is there a way to see what tables the nodes are storing? What would be the proper way to make changes to the config.ini file on a running system?

Thanks,

Jamie