Re: Data import speed

[Anton Aleksandrov]

Strange, that's a little bit too slow, really. We had similar situation - database with 22mln took 48 hours to import. Then I read an article which said - do not mix 1gbit with 100mbit, put either 1gbit either 100mbit on all boxes. I have tried that - those 22mln records went in 6 hours. Can you feel the difference? Load on my 1gbit network during import didn't get higher than 4Mbit in first try and 14Mbit in the second try.

Please also check that your db nodes don't use swap space a lot. While they are swapping -nothing is getting imported. On our API nodes I put the most simpliest my.cnf, from my.cnf-small (from support-files). And investigate your cluster log, may be there some additional info.

Oh.. And how many indexes do you have in that table? Putting as less indexes as you can (or even adding indexes when import is finished) could increase speed.