Load Data Infile takes a long time
I have an empty table which has about 7 columns. There are indexes on 6 of them. Also the primary key is the combination of all 6 of these columns.
I have also partitioned the table on the date column (ranged per month).
I have a file that has about 20 million records (805M in size). I am trying to execute a LOAD DATA INFILE operation through a php script. It took loads of time, so I split the file into chunks of 100K records. The first set of 100K files took like 20secs per file, however the later ones started slowing down a lot. I also could see that my database machine was getting I/O bounded a lot.
Is there a way that I can speed this whole process? In todays world, ideally 20mill records is not really a big deal right? Looks like I am missing something here. Any pointers would really help.
Subject
Views
Written By
Posted
Load Data Infile takes a long time
6477
March 15, 2013 04:44PM
2387
March 16, 2013 06:15PM
2185
March 17, 2013 11:39PM
2052
March 17, 2013 11:05PM
2067
March 18, 2013 03:04AM
1795
March 19, 2013 08:28PM
1689
March 20, 2013 12:25PM
1672
March 20, 2013 05:03PM
1626
March 26, 2013 11:06AM
Sorry, you can't reply to this topic. It has been closed.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion
of Oracle or any other party.