MySQL Forums
Forum List  »  InnoDB clusters

Group replication stopped on one node after rebootClusterFromCompleteOutage
Posted by: Maciej Pazurkiewicz
Date: January 19, 2022 09:24AM

Hello everyone!

In an effort to make our MySQL InnoDB Cluster setup more resilient, in particular auto-healing if possible, I am testing various fault scenarios on VMs.

One of the scenarios is cutting power off all the three nodes of the cluster. After the nodes are back, I log to the Shell and issue `dba.rebootClusterFromCompleteOutage()`. The cluster is up again but more often than not one node fails to join it. This is what I see in the command's output:

ERROR: A GTID set check of the MySQL instance at 'instance1' determined that it is missing transactions that were purged from all cluster members
WARNING: es-flow1:3366: RuntimeError: The instance 'instance1' is missing transactions that were purged from all cluster members.
NOTE: Unable to rejoin instance 'instance1' to the cluster but the dba.rebootClusterFromCompleteOutage() operation will continue.

What I see in `Cluster.status()` is:

"instance1": {
    "address": "instance1", 
    "instanceErrors": [
        "NOTE: group_replication is stopped."
    ], 
    "memberRole": "SECONDARY", 
    "memberState": "OFFLINE", 
    "mode": "R/O", 
    "readReplicas": {}, 
    "role": "HA", 
    "status": "(MISSING)", 
    "version": "8.0.24"
},

This sounds like a problem that calls for removing the instance from the cluster and adding it again to force re-provisioning. However, what helps is a plain restart of the service. Is this an expected behavior or is there something I could improve in the configuration of my cluster?

Thanks for your help in advance!

Options: ReplyQuote


Subject
Views
Written By
Posted
Group replication stopped on one node after rebootClusterFromCompleteOutage
408
January 19, 2022 09:24AM


Sorry, only registered users may post in this forum.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.