Clustering and multiple masters

[Derek Clarkson]

My work wants to achieve the following scenario:

* Multiple independent servers containing a full copy of the software and database.
* Complete replication between servers so that any server can view the full set of data.
* All servers acting as local database servers for physically local users.
* Each server having the ability to run without a connection to the rest and still be able to accept updates.
* Auto syncing of updates when a server reconnects to server group.

To look into this I've been playing with clustering. I setup two machines with each machine having a mysqld, ndbd and ndb_mgmd instance running. The config.ini lists each servers services.

I found that I could get this to sort of work, however updates where being lost depending on which ndbd got started first. The second would of course simply start acting as a backup and see the firsts data rather that loading up it's own data, collecting the changes and then updating the first ndbd instance. So it appears that clustering cannot handle a situation where a network of servers gets split and rejoined. Unless of course, my simulations have a flaw ;-)

Anyone have any ideas ?

Derek