MySQL Forums
Forum List  »  General

Importing CSV files of Twitter Data
Posted by: Kimberly Chulis
Date: February 10, 2015 02:16PM

I'm able to import raw Twitter archives by converting .xls files into .csv, then using the following code:

load data local infile 'filename.csv' into table filename
fields terminated by ','
;

The issue is that one of my fields is unstructured Tweet text data and occasionally users Tweet using commas, and in these instances this above command reads the comma as a field break and the data is read incorrectly for that entry. My original file does not contain quotes or any additional characters to distinguish fields, and I have upwards of 200 separate .xls and associated .csv files. Is there code to add to the load data infile statement that will skip any commas in one of the specified fields? Is there a better way to read these files in using the Python extension, that will provide a more efficient workaround?

Options: ReplyQuote


Subject
Written By
Posted
Importing CSV files of Twitter Data
February 10, 2015 02:16PM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.