MySQL Forums
Forum List  »  Full-Text Search

double-encoding of Arabic text
Posted by: Rick James
Date: August 25, 2012 01:18AM

http://bugs.mysql.com/bug.php?id=30277

FULLTEXT is designed for English, I suspect it has problems with other languages and other character sets.

Sorry, the text was double-encoded as it was stored. This will render any collation 'incorrect'. My document somewhat discusses how to fix the data (assuming it is not beyond repair).

SELECT HEX(CONVERT(CONVERT(UNHEX( 'C398C2AF') USING utf8) USING latin1));
--> D8AF, which is the utf8 for Arabic DAL.
SELECT CONVERT(CONVERT(UNHEX( 'C398C2AF') USING utf8) USING latin1);
--> that character.

Utf8 encodings for Basic Arabic characters:
http://lcweb2.loc.gov/diglib/codetables/33.html

Options: ReplyQuote


Subject
Views
Written By
Posted
double-encoding of Arabic text
2720
August 25, 2012 01:18AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.