Re: Cluster Crashes after

[Justin Lambert]

I had this exact same experience upgrading from 5.0.12 to 5.0.13. I wanted to see if a rolling upgrade would work so I first upgraded the management node, then each of the 4 storage nodes. I have replicas set to 2 as well so our storage node setup was about the same. The first 3 storage nodes upgraded fine and everything worked as expected and the databases were still available. On the 4th node I stopped the storage engine, updated mysql, and restarted the storage engine. I checked a bit later to make sure it finished syncing and found the cluster had crashed with the same error message. The only way I was able to bring it backup up is restart everything with --initial and reload the backup I had made prior to upgrading. I kept a copy of the mysql and mysql cluster directories in case they are useful when looking for a bug.