Re: uploading 300+MB files to MySQL db
If this is something you will do on a regular basis and you have some manipulation or validation you want to perform then you would have two scripts.
One that picks up the file and reads it row by row (CSV style), that then launches a second script which will perform your validation, manipulation and insert.
The script that parses the CSV file should have a configuration file or take as parameters some limiting value so as to not overwhelm the server it's running on.
In this way you can have as many 'feed' servers or processes as you need that push data to the backend.
You could start all 5 CSV's at once on a server or run each CSV on a seperate server or ...
Shawn
Subject
Written By
Posted
November 07, 2011 08:36PM
November 07, 2011 10:12PM
November 08, 2011 06:28PM
November 08, 2011 06:50PM
November 08, 2011 08:03PM
November 09, 2011 05:37PM
Re: uploading 300+MB files to MySQL db
November 09, 2011 09:08AM
November 09, 2011 05:39PM
November 12, 2011 02:09PM
November 17, 2011 05:18PM
Sorry, you can't reply to this topic. It has been closed.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion
of Oracle or any other party.