Re: High cpu load problem

[mike wylie]

The system works by importing all of the data to an import table first, then works through the import table digesting the information.

The digesting step is quite intensive as it inserts into a couple of tables, but it also does a number of selects to check for duplicates.

Once all of the data is digested, a second process selects every entry in the main table - as we then apply text indexing. This final step of selecting everything one by one seems to hammer mysql. It can take a couple of days to finish.

Any further thoughts on tweaks?

I have posted the system variables in reply to another user, this may spread some light?

Many thanks