innodb cluster kind of broken after reboot

[Damjan Kumin]

Hello friends,
I have set up 3 node HA innodb cluster. After initial configuration, everything worked as promised, no problems or inconstancies. Now I rebooted nodes via system, reboot now. Nodes came back online and after I had manually started router (different story altogether), I can see only this:

MySQL localhost:3306 ssl JS > cluster.status()
{
"clusterName": "innodb_cluster",
"defaultReplicaSet": {
"name": "default",
"primary": "mysql-master1:3306",
"ssl": "REQUIRED",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures. 2 members are not active.",
"topology": {
"mysql-master1:3306": {
"address": "mysql-master1:3306",
"memberState": "(MISSING)",
"mode": "n/a",
"readReplicas": {},
"role": "HA",
"shellConnectError": "MySQL Error 1045 (28000): Access denied for user 'root'@'mysql-master1.sidk.remote' (using password: YES)",
"status": "ONLINE",
"version": "8.0.23"
},
"mysql-master2:3306": {
"address": "mysql-master2:3306",
"mode": "n/a",
"readReplicas": {},
"role": "HA",
"shellConnectError": "MySQL Error 1045 (28000): Access denied for user 'root'@'mysql-master1.sidk.remote' (using password: YES)",
"status": "(MISSING)"
},
"mysql-master3:3306": {
"address": "mysql-master3:3306",
"mode": "n/a",
"readReplicas": {},
"role": "HA",
"shellConnectError": "MySQL Error 1045 (28000): Access denied for user 'root'@'mysql-master1.sidk.remote' (using password: YES)",
"status": "(MISSING)"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "mysql-master1:3306"
}
MySQL localhost:3306 ssl JS >

For the life of me I cannot get it back online as it is supposed to be. How do I get nodes connected - i designated account for interconnection and still it is trying to use root and ofc it is not connecting. Any help, please, would be greatly appreciated as I am now running on single non-fault-tolerant master.

Regards, Damjan



Edited 1 time(s). Last edit at 03/26/2021 06:31AM by Damjan Kumin.