Re: How to scale MySQL Cluster to support PB storage
1. Yes, I did look at VIPs and other load balancing solutions. As I said,
I do not expect much issues on this way, so I don't worry about this
right now.
2. You are right, the data need to be replicated in two different geo
locations to protect against failure of a data center. As far as I know,
MySQL does support this (in asynchronous mode).
At this time, I am considering about a sharding framework and in
particular if I should relay on replication feature of MySQL or have
this feature implemented as a part of the sharding framework.
Having automatic failover is a requirement, so if it can't be done using
MySQL replication, I will have to have it at the sharding level.
On the other hand, the sharding framework will do a lot with data
sharding/migration, so having the replication feature implemented on
this level is quite native.
What do you think?
3. I don't think that hash of the object name approach is an option in
my case. The point is that listing lexicographically consequent object
names must be supported, so preserving the objects names order is a
requirement. Assuming that hash will break the order, it can't be used.
The third option relays on hash and unlikely can be used as well. So,
the only remaining option is the dictionary.
4. Thanks for the idea to convert the object names to BIGINT. From my
perspective, it is the same as using the hash approach and, the same,
BIGINT's are fine, if an efficient approach of listing the object names
can be implemented.
5. I did not plan to visit MySQL Conference, but in any case I will let
you know how it turns out.
In fact, I am considering alternative approaches (non-RDBMS-based)
approach. At this time I am looking HyperTable and HBase (BigTable's
clones). They seems to provide all the features I need, but are quite
raw.
So, the final solution might have few in common with MySQL. ;)
BTW, do you have any opinion about Hbase and HyperTable?