Hi,
The way to increase speed is to ensure that the insert commands in the dump
insert many records per INSERT. I think you can specify this when you can create
the dump.
Rgrds Mikael
Adam D wrote:
> Ive been playing around with cluster quite a bit
> over the last week. I have found it quite good
> apart from the trouble I have with importing
> data;
>
> Import speed from previous myisam tables
> incredibly slow.A very old 1 month-only backup of
> 800,000 odd records from a table we have that now
> grows at 7000 records an hour took around 15hours
> (53,000 records per hour, and when your talking xx
> million records...)
>
> Just FYI the file is a mysqldump from a myisam
> table, where Ive changed the table creation to
> ndbcluster.
>
> I tried messing with the config.ini to make it
> 'try harder' in imports. Here are some of the
> variables set;
> MaxNoOfConcurrentOperations=10000
> MaxNoOfConcurrentTransactions=10000
> MaxNoOfLocalOperations=100000
> MaxNoOfAttributes = 10000
> MaxNoOfOrderedIndexes=5000
> MaxNoOfUniqueHashIndexes=5000
> MaxNoOfTables=250
>
> I thought maybe that increasing the amount of
> computers involved in datanodes that it might
> increase the speed capable, however the most I
> could get in 'Operations' on each of the nodes was
> around 400. Basically changing the above settings
> seemed to do next to nothing.
>
> The only way I could get the database 'driving'
> harder, was to dump multiple tables in at any one
> time. eg;
> mysql clusttest < dump1.sql &
> mysql clusttest < dump2.sql &
>
> That seemed to make it 'go harder' however still
> fairly average, I got it up to 1000 operations
> doing a few of them at once. I also tried to dump
> the table in to the database as myisam, which was
> quick as, then single mode alter table to make the
> engine ndbcluster. which took just as long.
>
> I checked network throughput, as each computer
> node is graphed, and they do not go over 600kbit.
> They are gigabit cards at 100 full
>
> Is there a way I can increase the speed at which
> the cluster inserts records.
Mikael Ronstrom
Senior Software Architect, MySQL AB
My blog:
http://mikaelronstrom.blogspot.com