Cluster not working anymore

[Jan D]

Hello together,

this morning our cluster started to behave very weird.
Not sure if its related to the changes I made yesterday/this morning.
Reverted all changes though and rebooted the servers, didnt change anything though.

The cluster runs on two win 2008 servers.
The changes I made yesterday are:

[HKEY_LOCAL_MACHINE \System \CurrentControlSet \Services \Tcpip \Parameters]
MaxFreeTcbs = 20000
MaxHashTableSize = 1024
KeepAliveTime = 1800000
MaxUserPort = 65534
TcpTimedWaitDelay = 120

and

[mysqld]
interactive_timeout=180
wait_timeout=180
max_connections=1000

as said I reverted them already, without luck.


The "weird" behavior shows as followed:


Fetching tables in workbench takes a minute,
before those are not fetched, every sql query will take forever and workbench freezes
when they are fetched sql queries will work, but they are awefully slow, weird here is,
that it shows me 0.031 dutation in the output window, 0 seconds to fetch, but it takes nearly a minute till the data shows in workbench.
SQL Queries that are done programmatically by our apps run fine though..so it seems workbench is having issues for some reason.

Above applies for one server, on the other all fails, queries in workbench doesnt work, neither dong them programmatically.


So I ran mysqlcheck on both servers.
On one it told me that all my tables are "read only"
additionally running "Show full processlist" tells me

1, system user, , , Daemon, 3957, Waiting for ndbcluster to start,
5, root, localhost:49276, kunden, Query, 3435, Opening tables, SHOW INDEXES FROM xxxx
7, root, localhost:49407, kunden, Query, 2974, Opening tables, SHOW INDEXES FROM xxxx
14, root, localhost:49665, kunden, Sleep, 1460, ,
19, xxxx, xxxxx, kunden, Query, 1816, statistics, SELECT * FROM xxxx
20, xxxx, xxxxx, kunden, Query, 1749, statistics, SELECT * FROM xxxx

..so all queries seem to stuck and the first line seems to indcate, that the ndbcluster is not running at all.

running "show" on ndb_mgm.exe shows me several times that my the mysqld(API) node is one time not connected, one time it is.


The other server is fine, no read only tables, no stucked queries in full processlist.


I restarted the server a couple of times, no change.
The tables stay in read only mode and the ndbcluster seems not to run.

Any help is much appreciated, thanks in advance.