Error 2327: Memory allocation failure
Sorry to be such a bother.
We're trying to bring up a new node in our cluster to replace one on another computer which has become unstable. After bringing the other computer down and swapping out IP addresses, we try to bring up the new node, only to constantly run into this error:
-----
Node 3: Forced node shutdown completed. Occured during startphase 0. Initiated by signal 0. Caused by error 2327: 'Memory allocation failure, please decrease some configuration parameters(Configuration error). Permanent error, external action needed'.
-----
We're running MySQL 5.0.18 across four machines. This is our basic configuration (nodes 3 and 9 are the machine we are trying to replace):
-----
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 4 node(s)
id=3 (not connected, accepting connect from x.x.x.58)
id=4 @x.x.x.59 (Version: 5.0.18, Nodegroup: 0, Master)
id=5 @x.x.x.60 (Version: 5.0.18, Nodegroup: 1)
id=6 @x.x.x.61 (Version: 5.0.18, Nodegroup: 1)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @x.x.x.57 (Version: 5.0.18)
[mysqld(API)] 4 node(s)
id=9 (not connected, accepting connect from x.x.x.58)
id=10 @x.x.x.59 (Version: 5.0.18)
id=11 @x.x.x.11 (Version: 5.0.18)
id=12 @x.x.x.2 (Version: 5.0.18)
ndb_mgm> all status
Node 3: not connected
Node 4: started (Version 5.0.18)
Node 5: started (Version 5.0.18)
Node 6: started (Version 5.0.18)
-----
We can get it to start running if we decrese the memory allocations in my.conf to 512MB for tables and 512MB for indexes. All of our other machines are running 768MB for tables, so we need this one to match those.
I remember once running into some docs about a Linux kernel setting which determined the maximum size of a single file that can be opened in memory, and that this needs to be increased in order for the database to grow larger.I can't seem to find anything about this anymore, though. Curse my spotty memory. ;-)
The new machine we are trying to swap in is a hardened Gentoo box with gr-security applied. The old ones are Slackware 8, also with gr-security applied.
Thank you all very much for any assistance you can provide.
Regards,
Darren Landrum