How to deal with foreign characters using MySql within Java EE environment

Question

I want to read data from a csv file and then write to MySql. The data contains foreign Languages.

I got this error when I tried to insert a record, which contains Japanese Characters, into MySql.

"1366Incorrect string value: '\xE6\xB0\xB4\xE7\x9D\x80...' for column 'name' at row 1"

The SQL sentence looks like this:

INSERT INTO `MerchandiseMaster` (id,name) VALUES ('20000101','JANIE AND JACK水着　鶯茶系　大胆花柄')

My csv file uses UTF-8 Encoding and the charset of MySql database schema is utf8_gerneral_ci .

I have put these parameters when I connect to database through JDBC( mysql-connector-java-5.1.34-bin.jar ):

connect = DriverManager.getConnection("jdbc:mysql://localhost/mydata?"
    + "useUnicode=yes&characterEncoding=UTF-8&user=user123&password=user123.");

My question is:

Is there anything else that I am missing to deal with foreign characters correctly?

Answer 1

I found this on a website, so caveat emptor , but apparently MySQL's UTF-8 support is incomplete. In 2010 they added new support, utf8mb4 that supports the entire UTF-8 encoding scheme.

Add to your MySQL configuration file:

[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

Here's a link to the full article. I haven't tried this out, so test everything carefully first, and make a back-up of your database before doing anything.