MySQL Node not Connecting to Management Nodes in NDB Cluster
Posted by: Mahsa Molayi
Date: August 24, 2018 05:08PM
Date: August 24, 2018 05:08PM
Hi,
I am trying to deploy a 5 machine MySQL NDB Cluster with 1 Management Nodes, 2 SQL Nodes and 2 Data Nodes. Running everything on amazon Ec2 instances.
I have defined all hosts with their public hostname and public IPs in /etc/hosts
for instance /ect/hosts in one mysql node is :
127.0.0.1 ec2-54-202-3-137.us-west-2.compute.amazonaws.com.
::1 localhost6 localhost6.localdomain6
54.202.3.137 ec2-54-202-3-137.us-west-2.compute.amazonaws.com.
52.10.163.138 ec2-52-10-163-138.us-west-2.compute.amazonaws.com.
52.90.65.179 ec2-52-90-65-179.compute-1.amazonaws.com.
54.209.4.25 ec2-54-209-4-25.compute-1.amazonaws.com
34.222.50.157 ec2-34-222-50-157.us-west-2.compute.amazonaws.com.
My config.ini file reads as follow:
[ndb_mgmd default]
# Directory for MGM node log files
DataDir=/var/lib/mysql-cluster
[ndb_mgmd]
#Management Node db1
HostName=ec2-52-10-34-29.us-west-2.compute.amazonaws.com.
[ndbd default]
NoOfReplicas=2 # Number of replicas
DataMemory=80M # Memory allocate for data storage
IndexMemory=10M # Memory allocate for index storage
ServerPort=2202
#Directory for Data Node
DataDir=/var/lib/mysql-cluster
[ndbd]
#Data Node db2
HostName=ec2-34-222-50-157.us-west-2.compute.amazonaws.com.
[ndbd]
#Data Node db3
HostName=ec2-52-90-65-179.compute-1.amazonaws.com.
[mysqld]
#SQL Node db4
HostName=ec2-54-202-3-137.us-west-2.compute.amazonaws.com
[mysqld]
#SQL Node db5
HostName=ec2-54-209-4-25.compute-1.amazonaws.com.
On the MySQL node the my.cnf file reads as:
[mysqld]
ndbcluster
#ndb-connectstring=ec2-52-10-34-29.us-west-2.compute.amazonaws.com. # IP address for server management node
default_storage_engine=ndbcluster # Define default Storage Engine used by MySQL
bind-address = 0.0.0.0
#[mysql_cluster]
#ndb-connectstring=ec2-52-10-34-29.us-west-2.compute.amazonaws.com. # IP address for server management node
On starting the Management nodes, both come online fine however when I start the MySQL node, it does not see the Management nodes.
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @34.222.50.157 (mysql-5.6.28 ndb-7.4.10, Nodegroup: 0, *)
id=3 @52.90.65.179 (mysql-5.6.28 ndb-7.4.10, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @52.10.34.29 (mysql-5.6.28 ndb-7.4.10)
[mysqld(API)] 2 node(s)
id=4 (not connected, accepting connect from ec2-54-202-3-137.us-west-2.compute.amazonaws.com)
id=5 (not connected, accepting connect from ec2-54-209-4-25.compute-1.amazonaws.com.)
my data nodes are connected but when I start my sql node with this command I have this log file:
2018-08-24 22:54:35 4190 [Warning] Buffered warning: Changed limits: max_open_files: 1024 (requested 5000)
2018-08-24 22:54:35 4190 [Warning] Buffered warning: Changed limits: table_open_cache: 431 (requested 2000)
2018-08-24 22:54:35 4190 [Note] Plugin 'FEDERATED' is disabled.
2018-08-24 22:54:35 4190 [Note] InnoDB: Using atomics to ref count buffer pool pages
2018-08-24 22:54:35 4190 [Note] InnoDB: The InnoDB memory heap is disabled
2018-08-24 22:54:35 4190 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-08-24 22:54:35 4190 [Note] InnoDB: Memory barrier is not used
2018-08-24 22:54:35 4190 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-08-24 22:54:35 4190 [Note] InnoDB: Using Linux native AIO
2018-08-24 22:54:35 4190 [Note] InnoDB: Using CPU crc32 instructions
2018-08-24 22:54:35 4190 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-08-24 22:54:35 4190 [Note] InnoDB: Completed initialization of buffer pool
2018-08-24 22:54:35 4190 [Note] InnoDB: Highest supported file format is Barracuda.
2018-08-24 22:54:35 4190 [Note] InnoDB: The log sequence numbers 1626057 and 1626057 in ibdata files do not match the log sequence number 1626067 in the ib_logfiles!
2018-08-24 22:54:35 4190 [Note] InnoDB: Database was not shutdown normally!
2018-08-24 22:54:35 4190 [Note] InnoDB: Starting crash recovery.
2018-08-24 22:54:35 4190 [Note] InnoDB: Reading tablespace information from the .ibd files...
2018-08-24 22:54:35 4190 [Note] InnoDB: Restoring possible half-written data pages
2018-08-24 22:54:35 4190 [Note] InnoDB: from the doublewrite buffer...
2018-08-24 22:54:35 4190 [Note] InnoDB: 128 rollback segment(s) are active.
2018-08-24 22:54:35 4190 [Note] InnoDB: Waiting for purge to start
2018-08-24 22:54:35 4190 [Note] InnoDB: 5.6.28 started; log sequence number 1626067
2018-08-24 22:54:35 4190 [Note] NDB: Changed global value of binlog_format from STATEMENT to MIXED
2018-08-24 22:54:35 4190 [Note] NDB: NodeID is 4, management server 'ec2-52-10-34-29.us-west-2.compute.amazonaws.com.:1186'
2018-08-24 22:55:05 4190 [Note] NDB[0]: NodeID: 4, no storage nodes connected (timed out)
2018-08-24 22:55:05 4190 [Warning] NDB: server id set to zero - changes logged to bin log with server id zero will be logged with another server id by slave mysqlds
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Starting...
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Started
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Setting up
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Created schema Ndb object, reference: 0x80040004, name: 'Ndb Binlog schema change monitoring'
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Created injector Ndb object, reference: 0x80050004, name: 'Ndb Binlog data change monitoring'
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Setup completed
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Wait for server start completed
2018-08-24 22:55:05 4190 [Note] NDB Util: Starting...
2018-08-24 22:55:05 4190 [Note] NDB Util: Wait for server start completed
2018-08-24 22:55:05 4190 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
2018-08-24 22:55:05 4190 [Note] - '0.0.0.0' resolves to '0.0.0.0';
2018-08-24 22:55:05 4190 [Note] Server socket created on IP: '0.0.0.0'.
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Starting...
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Wait for server start completed
2018-08-24 22:55:05 4190 [Note] Event Scheduler: Loaded 0 events
2018-08-24 22:55:05 4190 [Note] mysqld: ready for connections.
Version: '5.6.28-ndb-7.4.10-cluster-gpl' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Cluster Community Server (GPL)
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Check for incidents
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Wait for cluster to start
2018-08-24 22:55:05 4190 [Note] NDB Util: Wait for cluster to start
2018-08-24 22:55:05 4190 [Note] NDB Util: Started
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Wait for cluster to start
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Started
2018-08-24 22:55:05 4190 [Note] created index stats Ndb object: reference 0x80070004, name: 'Ndb Index Statistics monitoring'
2018-08-24 22:55:35 4190 [Warning] NDB : Tables not available after 30 seconds. Consider increasing --ndb-wait-setup value
Any pointers will be greatly appreciated.
I am trying to deploy a 5 machine MySQL NDB Cluster with 1 Management Nodes, 2 SQL Nodes and 2 Data Nodes. Running everything on amazon Ec2 instances.
I have defined all hosts with their public hostname and public IPs in /etc/hosts
for instance /ect/hosts in one mysql node is :
127.0.0.1 ec2-54-202-3-137.us-west-2.compute.amazonaws.com.
::1 localhost6 localhost6.localdomain6
54.202.3.137 ec2-54-202-3-137.us-west-2.compute.amazonaws.com.
52.10.163.138 ec2-52-10-163-138.us-west-2.compute.amazonaws.com.
52.90.65.179 ec2-52-90-65-179.compute-1.amazonaws.com.
54.209.4.25 ec2-54-209-4-25.compute-1.amazonaws.com
34.222.50.157 ec2-34-222-50-157.us-west-2.compute.amazonaws.com.
My config.ini file reads as follow:
[ndb_mgmd default]
# Directory for MGM node log files
DataDir=/var/lib/mysql-cluster
[ndb_mgmd]
#Management Node db1
HostName=ec2-52-10-34-29.us-west-2.compute.amazonaws.com.
[ndbd default]
NoOfReplicas=2 # Number of replicas
DataMemory=80M # Memory allocate for data storage
IndexMemory=10M # Memory allocate for index storage
ServerPort=2202
#Directory for Data Node
DataDir=/var/lib/mysql-cluster
[ndbd]
#Data Node db2
HostName=ec2-34-222-50-157.us-west-2.compute.amazonaws.com.
[ndbd]
#Data Node db3
HostName=ec2-52-90-65-179.compute-1.amazonaws.com.
[mysqld]
#SQL Node db4
HostName=ec2-54-202-3-137.us-west-2.compute.amazonaws.com
[mysqld]
#SQL Node db5
HostName=ec2-54-209-4-25.compute-1.amazonaws.com.
On the MySQL node the my.cnf file reads as:
[mysqld]
ndbcluster
#ndb-connectstring=ec2-52-10-34-29.us-west-2.compute.amazonaws.com. # IP address for server management node
default_storage_engine=ndbcluster # Define default Storage Engine used by MySQL
bind-address = 0.0.0.0
#[mysql_cluster]
#ndb-connectstring=ec2-52-10-34-29.us-west-2.compute.amazonaws.com. # IP address for server management node
On starting the Management nodes, both come online fine however when I start the MySQL node, it does not see the Management nodes.
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @34.222.50.157 (mysql-5.6.28 ndb-7.4.10, Nodegroup: 0, *)
id=3 @52.90.65.179 (mysql-5.6.28 ndb-7.4.10, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @52.10.34.29 (mysql-5.6.28 ndb-7.4.10)
[mysqld(API)] 2 node(s)
id=4 (not connected, accepting connect from ec2-54-202-3-137.us-west-2.compute.amazonaws.com)
id=5 (not connected, accepting connect from ec2-54-209-4-25.compute-1.amazonaws.com.)
my data nodes are connected but when I start my sql node with this command I have this log file:
2018-08-24 22:54:35 4190 [Warning] Buffered warning: Changed limits: max_open_files: 1024 (requested 5000)
2018-08-24 22:54:35 4190 [Warning] Buffered warning: Changed limits: table_open_cache: 431 (requested 2000)
2018-08-24 22:54:35 4190 [Note] Plugin 'FEDERATED' is disabled.
2018-08-24 22:54:35 4190 [Note] InnoDB: Using atomics to ref count buffer pool pages
2018-08-24 22:54:35 4190 [Note] InnoDB: The InnoDB memory heap is disabled
2018-08-24 22:54:35 4190 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-08-24 22:54:35 4190 [Note] InnoDB: Memory barrier is not used
2018-08-24 22:54:35 4190 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-08-24 22:54:35 4190 [Note] InnoDB: Using Linux native AIO
2018-08-24 22:54:35 4190 [Note] InnoDB: Using CPU crc32 instructions
2018-08-24 22:54:35 4190 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-08-24 22:54:35 4190 [Note] InnoDB: Completed initialization of buffer pool
2018-08-24 22:54:35 4190 [Note] InnoDB: Highest supported file format is Barracuda.
2018-08-24 22:54:35 4190 [Note] InnoDB: The log sequence numbers 1626057 and 1626057 in ibdata files do not match the log sequence number 1626067 in the ib_logfiles!
2018-08-24 22:54:35 4190 [Note] InnoDB: Database was not shutdown normally!
2018-08-24 22:54:35 4190 [Note] InnoDB: Starting crash recovery.
2018-08-24 22:54:35 4190 [Note] InnoDB: Reading tablespace information from the .ibd files...
2018-08-24 22:54:35 4190 [Note] InnoDB: Restoring possible half-written data pages
2018-08-24 22:54:35 4190 [Note] InnoDB: from the doublewrite buffer...
2018-08-24 22:54:35 4190 [Note] InnoDB: 128 rollback segment(s) are active.
2018-08-24 22:54:35 4190 [Note] InnoDB: Waiting for purge to start
2018-08-24 22:54:35 4190 [Note] InnoDB: 5.6.28 started; log sequence number 1626067
2018-08-24 22:54:35 4190 [Note] NDB: Changed global value of binlog_format from STATEMENT to MIXED
2018-08-24 22:54:35 4190 [Note] NDB: NodeID is 4, management server 'ec2-52-10-34-29.us-west-2.compute.amazonaws.com.:1186'
2018-08-24 22:55:05 4190 [Note] NDB[0]: NodeID: 4, no storage nodes connected (timed out)
2018-08-24 22:55:05 4190 [Warning] NDB: server id set to zero - changes logged to bin log with server id zero will be logged with another server id by slave mysqlds
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Starting...
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Started
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Setting up
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Created schema Ndb object, reference: 0x80040004, name: 'Ndb Binlog schema change monitoring'
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Created injector Ndb object, reference: 0x80050004, name: 'Ndb Binlog data change monitoring'
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Setup completed
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Wait for server start completed
2018-08-24 22:55:05 4190 [Note] NDB Util: Starting...
2018-08-24 22:55:05 4190 [Note] NDB Util: Wait for server start completed
2018-08-24 22:55:05 4190 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
2018-08-24 22:55:05 4190 [Note] - '0.0.0.0' resolves to '0.0.0.0';
2018-08-24 22:55:05 4190 [Note] Server socket created on IP: '0.0.0.0'.
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Starting...
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Wait for server start completed
2018-08-24 22:55:05 4190 [Note] Event Scheduler: Loaded 0 events
2018-08-24 22:55:05 4190 [Note] mysqld: ready for connections.
Version: '5.6.28-ndb-7.4.10-cluster-gpl' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Cluster Community Server (GPL)
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Check for incidents
2018-08-24 22:55:05 4190 [Note] NDB Binlog: Wait for cluster to start
2018-08-24 22:55:05 4190 [Note] NDB Util: Wait for cluster to start
2018-08-24 22:55:05 4190 [Note] NDB Util: Started
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Wait for cluster to start
2018-08-24 22:55:05 4190 [Note] NDB Index Stat: Started
2018-08-24 22:55:05 4190 [Note] created index stats Ndb object: reference 0x80070004, name: 'Ndb Index Statistics monitoring'
2018-08-24 22:55:35 4190 [Warning] NDB : Tables not available after 30 seconds. Consider increasing --ndb-wait-setup value
Any pointers will be greatly appreciated.
Subject
Views
Written By
Posted
MySQL Node not Connecting to Management Nodes in NDB Cluster
1725
August 24, 2018 05:08PM
685
August 27, 2018 07:57AM
Sorry, you can't reply to this topic. It has been closed.
Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.