question marks on insert of Unicode Japanese text with Perl
Posted by: Dan McCormick
Date: August 10, 2005 05:20PM

Hi,

When I try to insert Unicode Japanese text into a column, it gets converted to question marks. I have a test table, t, with one column, kw:

mysql> select hex(kw), hex("?") from t;
+---------+----------+
| hex(kw) | hex("?") |
+---------+----------+
| 3F3F3F | 3F |
+---------+----------+

The table was created to support UTF-8:

create table t ( kw varchar(255) character set utf8 collate utf8_unicode_ci NOT NULL default '' ) ENGINE=MyISAM DEFAULT CHARSET=utf8;

I'm inserting via a Perl script. I first set the character set:

$dbh->do("set character set utf8");

I then get a string from a web site, and Devel::Peek::Dump assures me that it's Unicode:

SV = PV(0x953b584) at 0x91309f8
REFCNT = 1
FLAGS = (PADBUSY,PADMY,POK,pPOK,UTF8)
PV = 0x94f8c80 "\343\202\253\343\203\251\343\203\274"\0 [UTF8 "\x{30ab}\x{30e9}\x{30fc}"]
CUR = 9
LEN = 12

I can also use Unicode::CharName to look at each of the three characters that are returned:

KATAKANA LETTER KA
KATAKANA LETTER RA
KATAKANA-HIRAGANA PROLONGED SOUND MARK

I then do a simple insert:
my $sth = $dbh->prepare("insert into t (kw) values (?)");
$sth->execute( $t );

and end up with the set of three question marks shown above.

I'm using mysql 4.1.13, Perl 5.8.5, and DBD::mysql 3.0002.

Any advice would be greatly appreciated.

Thanks,
Dan

Options: ReplyQuote


Subject
Views
Written By
Posted
question marks on insert of Unicode Japanese text with Perl
5262
August 10, 2005 05:20PM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.