Re: How to scale MySQL Cluster to support PB storage

[Rick James]

Automatic failover -- virtually impossible. There are too many things that can fool the automatic system. You don't want to ever have both masters thinking they are alone -- something that can happen if there is a partial network outage.

Each shard should have 2 masters, one of which is being written to. The masters should be in different geo-locations and be replicating from each other. For maximal ease of recovery from certain disasters, also have at least one slave hanging off each master.

Read scaling: add slaves.

Write scaling: add shards.

I'm ignorant of Hbase and HyperTable. Be cautious -- an abstraction layer may or may not abstract the right things. An abstraction that does single row lookups is good if that is what you are doing. A layer that hashes the items is clumsy when you ask for a range. JOINs are either not handled or have performance that would make you want to punt. Etc.

Given that you need to fetch a "range" of items by key, BIGINT and Hash are impractical (as you say).

My "in between -- hash to a 16-bit value, then do dictionary lookup"...
Well, replace the hashing with a roughly even partitioning of the lexically ordered keys into several thousand buckets. Depending on the keys, this might work. (For something like timestamps, there are always new buckets coming into existence. And possibly old ones being purged. Doable, but not straight forward.)

Good Luck.