MySQL Forums
Forum List  »  Connector/Node.js

fast XML storage into MySQL
Posted by: FRANCOIS guerin
Date: May 22, 2009 08:28AM


I'd like to insert about 2 billions of XML files (4-8 Ko each) into my database.
Treating XML as text (then accessing its content with the ExtractValue function) is very fast (about 25000 files per minute), but as i said you can only access its content and the possible treatments are quite limited - you cant just access to a portion of a stored xml file...

So, after that, i tried to see my XML as tables and associations, so i modelized a schema. To parse all my xml files, i use script languages (python / perl), those are connected to MySQL with the appropriate drivers, and insert datas into my database,
but this solution is quite slow because i can only parse 3000 files per minute...

Does someone have a solution?
ie to store all my xml files in a litte time (more than 20000 files per minute) and then access to the datas (content and xml parts quite) in an easy/practical way?

Thanks a lot!



Options: ReplyQuote

Written By
fast XML storage into MySQL
May 22, 2009 08:28AM

Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.