Re: Can We Force MySQL to Terminate on Disk I/O Error?

[Authorized User]

Peter Brawley Wrote:
-------------------------------------------------------
> What problem is your proposal trying to solve?

Our clustering services have a dependency tree. From lowest to highest, it goes: DRBD -> FileSystem -> MySQL. If a failover condition is triggered, the cluster manager wants to (a) stop the services in reverse order, (b) fail the DRBD volume over to the secondary server, then (c) start the services on the secondary node.

The problem is that MySQL may refuse to stop in a timely fashion if it cannot write to disk. Eventually the service script times out, leaving the mysqld process running, with handles to the filesystem. Because of that, the filesystem will not unmount, and that prevents DRBD from failing over. Basically, if MySQL refuses to terminate when asked, it can prevent the whole cluster from recovering.

The traditional way to handle this problem is through cluster node fencing. If something goes wrong during the stopping process, then the whole cluster node gets rebooted. This is NOT an option in our environment. We need MySQL to commit seppuku instantly if disk IO errors so the rest of the cluster stack can fail over properly.