innodb clusters - after adding instances, their status from 'recovering' becomes 'missing'
Hi,
after adding 2 mysql instances (docker), their status becomes 'missing' as shown below:
mysql-js> cluster.status();
{
"clusterName": "isc",
"defaultReplicaSet": {
"name": "default",
"primary": "mysql-pri:3306",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures. 2 members are not active",
"topology": {
"mysql-pri:3306": {
"address": "mysql-pri:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"mysql-secondary1:3307": {
"address": "mysql-secondary1:3307",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "(MISSING)"
},
"mysql-secondary2:3308": {
"address": "mysql-secondary2:3308",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "(MISSING)"
}
}
}
}
I went into the mysql-secondary1 and the slave status:
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: mysql-pri
Master_User: root
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 15016
Relay_Log_File:
328e465c903d-relay-bin-mysql@002dpri.000002
Relay_Log_Pos: 398
Relay_Master_Log_File: mysql-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1146
Last_Error: Error executing row event: 'Table 'mysql_innodb_cluster_metadata.clusters' doesn't exist'
Skip_Counter: 0
Exec_Master_Log_Pos: 514
Relay_Log_Space: 16136
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1146
Last_SQL_Error: Error executing row event: 'Table 'mysql_innodb_cluster_metadata.clusters' doesn't exist'
Replicate_Ignore_Server_Ids:
Master_Server_Id: 11
Master_UUID: d3d3dd25-1062-11e7-b90d-0242c0a8032b
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp: 170325 12:29:45
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 4d4b7df6-10e3-11e7-b3af-0242c0a8032b:2-49
Executed_Gtid_Set: 4d4b7df6-10e3-11e7-b3af-0242c0a8032b:1,
d3d3dd25-1062-11e7-b90d-0242c0a8032b:1-3
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name: mysql-pri
Master_TLS_Version:
the mysql-secondary1 my.cnf is as below:
[mysqld]
port=3307
server-id=12
log-error = /var/lib/mysql/mysql.error
datadir=/var/lib/mysql
max_allowed_packet = 100M
lower_case_table_names=1
max_connections = 300
enforce_gtid_consistency = on
gtid_mode = on
log_bin = 328e465c903d-bin
log_slave_updates = true
binlog_checksum = none
master_info_repository = TABLE
relay_log_info_repository = TABLE
transaction_write_set_extraction = XXHASH64
relay-log = 328e465c903d-relay-bin
any help?
many thanks in advance