Processing unicode using Python and mysqldb

[Alvin Chin]

Hi,

I've been having lots of problems trying to process unicode using Python and mysqldb and I'm not sure what to do. I have some text that I want to put into a mysdl database but it keeps giving me the error:

UnicodeEncodeError: 'latin-1' codec can't encode character u'\ufeff' in position 12: ordinal not in range(256)

I've tried changing my database and table to be in latin-1 format, tried utf8_general_ci and utf8_unicode_ci. I have read articles saying you have to convert the actual character into unicode, but I have so much text that I'm grabbing from an online web site that the text could be anything and I just want to store it in a mysql database. I've tried even adding the use_unicode="True" to the connect statement in python. I've pretty much almost given up on what else to do.

Can someone please help? Thanks.

Alvin