Re: poor performance on load data to NDB

[Andy Kwong]

Thanks for your reply, Kevin,

On the network, I got each servers connected to a private gigabit network with 2 nics binded together. So, it is sort of a 2gb ethernet. I can copy the source data file (150MB) between the servers in seconds. I'm guessing the NDB engine maybe forcing the buffer to be send over to other ndb node too often (if not on row level). I will check out the ndb_force_send variable, but my worry is that it will delay the sync operation and the cluster is not as 'closely synced' as they designed to be.

I was hoping there is a way to:
- load data into one node temporary on temp table
- commit the bulk data change over the network (temp table)
- rename the temp table to replace a production table and the new data will get online on both node.
However, my first step take very long.

I tested an alternative by stopped all the node except one and I load the data on that node first. After that, I start the rests of the nodes and they sync up with the master. The total time is much faster than (the time it needs for a single node). However, there should be a way to do that without stopped the nodes.

Maybe I should have interface with the NDB engine directly to load data (not through the mysqld)?

Regards,
Andy



Edited 1 time(s). Last edit at 10/06/2006 09:34AM by Andy Kwong.