Re: Load Data Infile takes a long time

[Rick James]

How much RAM do you have? 57G for the buffer_pool is pretty tight for 64GB of RAM.

How many PARTITIONs are in that table? About 200? Beyond about 50, PARTITIONing had certain problems, such as opening all partitions to do just about anything.

What is the value of table_open_cache?

That's an awfully big primary key. Is that the minimal set of fields to provide uniqueness? It may be a lot better to switch to an AUTO_INCREMENT for the PK. Then have that monster as a secondary key, if you have any use for it (which I doubt).

Let's see the SELECTs that typically hit the table.

Each secondary index has a copy of the PK in it -- notice from SHOW TABLE STATUS how big the Index_length is. (Switching to AUTO_INC will shrink that tremendously.)

> KEY `gender_id` (`gender_id`),
Do not INDEX fields with only a small number of values -- the index will never be used. (There are rare exceptions.)

> `gender_id` int(11) NOT NULL DEFAULT '0',
That's 4 bytes to represent two values? Switch to TINYINT or CHAR(1); either is only 1 byte.
Don't normalize something (like gender) that can be stored in the table in the about the same amount of space.

Do not start any index with the PARTITION key (date) -- it eliminates the utility of "partition pruning". (There are exceptions.)

Do all the ALTERs at once.

And, yes, Øystein's suggestion will help you with the ALTER I am suggesting.