MySQL Forums
Forum List  »  Partitioning

horizontal partitioning on multiple nodes
Posted by: Nathan Hoult
Date: October 05, 2007 08:54AM

We have a need for a 250 gig table with only 2 columns, key and binary blob. We will be doing 99% selects but will be mostly random access. We have looked at a number of commercial solutions, but found that none of them are fast enough and provide more than we need (don't 'have' to have complex queries, multi index...). We have come to the conclusion that disk latency are the limiting factor. We would be willing to buy 250gigs worth of memory in multiple 64gig 64bit nodes, and solid state drives when they are supported by our hardware provider (2 years?). I have been searching with Google for around 16 hours, but the closest thing I have found is how to cluster MySQL that mirrors the data on multiple nodes. I also found how to make MySQL 5.1 do horizontal partitioning (lead me here), but it seems to just be disk based partitioning.
In summery was wondering if MySQL can:
- In memory engine that supports BLOBS (so far my searching has said no, has that changed?). Disk based that after a while of selecting, will all be cached in memory is an option.
- Use a hash or range to put the data on other nodes, any node I insert to should put it on the right node, and any node I select from should get it from the right node. (aka: client does not need to managed the distribution)
- if a node has a problem, some way to block inserts/selects until the node is restored. It should maintain the data through a reboot, a hard drive loss could require a cluster reload.

We currently have an in house solution that does not support live updates and causes us problems, sadly it currently beats every commercial distributed Database system we have tested by 4x speed or more. We are trying to avoid another in house solution as we have to support it.

Thanks in advance.

Options: ReplyQuote

Written By
horizontal partitioning on multiple nodes
October 05, 2007 08:54AM

Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.