Java POST数据到mySQL UTF-8编码问题

Question

I have POST data that contains the Japanese string AKB48 ネ申テレビ シーズン3 , defined in jQuery as data .

$("#some_div").load("someurl", { data : "AKB48 ネ申テレビ シーズン3"}) 

The post data is sent to Java Servlet:

String data = new String(this.request.getParameter("data").getBytes("ISO-8859-1"), "UTF-8");

My program saves it to MySQL, but after the data is saved to the database it becomes:

AKB48 u30CDu7533u30C6u30ECu30D3 u30B7u30FCu30BAu30F33

What should I do if I want to save it as it is in UTF-8? All my files are in UTF-8.

MySQL encoding is utf8 and here is the code

String sql = "INSERT INTO Inventory (uid, item_id, item_data, ctime) VALUES ("
                + inventory.getUid() + ",'"
                + inventory.getItemId() + "','"
                + StringEscapeUtils.escapeJava(inventory.getItemData()) + "',CURRENT_TIMESTAMP)";
    Statement stmt = con.createStatement();
    int cnt = stmt.executeUpdate(sql);

Answer 1

From your example above, I can verify that the Japanese string is getting saved to your MySQL database correctly, but as escaped Unicode .

I would check these items in order:

1. Are your tables and columns all set to have a character set and collation for utf8? Ie, CHARACTER SET utf8 COLLATE utf8_general_ci
2. Are explicitly setting the character set encoding before POST? request.setCharacterEncoding("UTF-8");
3. Are you setting the character encoding for your db connections? Ie, jdbc:mysql://localhost:3306/YOURDB?useUnicode=true&characterEncoding=UTF8

As the others have pointed out, you should not use that getBytes trick. It will surely mess up the POSTed values.

EDIT

Do not use StringEscapeUtils.escapeJava , since that will turn your string into escaped Unicode. That is what is transforming AKB48 ネ申テレビ シーズン3 into AKB48 u30CDu7533u30C6u30ECu30D3 u30B7u30FCu30BAu30F33 .

Answer 2

Why you do not just extract value of parameter like this.request.getParameter("data") ?

Your data is sent correctly using URL encoding where each unicode character is replaced by its code. Then you have to get the value of the parameter. When you are requesting bytes using ISO-8859-1 you are actually corrupting your data because the string is represented as a sequence if codes in textual form.

Answer 3

What's the point of the line

String data = new String(this.request.getParameter("data").getBytes("ISO-8859-1"), "UTF-8");

You're transforming chinese (or at least non-occidental) characters into bytes using the ISO-8859-1 encoding. Of course this can't work, since chinese characters are not supported by the ISO-8859-1 encoding. ANd then you're constructing a new String from bytes that are supposed to represent ISO-8859-1-encoded characters, using the UTF-8 encoding. This, once again, doesn't make any sense. UTF-8 and ISO-8859-1 are not the same thing, and only a small set of chars have the same encoding in both formats.

Just use

String data = this.request.getParameter("data");

and everything should be OK, provided that the column in the MySQL table uses an encoding that supports these characters.

EDIT:

now that you've shown us the code used to insert the data in database, I know where all this comes from (the preceding points are still valid, though). You're doing

StringEscapeUtils.escapeJava(inventory.getItemData())

What's the point? escapeJava is used to take a String and escape special characters in order to make it a valid Java String literal. It has nothing to do with SQL. Use a prepared statement:

String sql = "INSERT INTO Inventory (uid, item_id, item_data, ctime) VALUES (?, ?, ?, CURRENT_TIMESTAMP);
PreparedStatement stmt = con.prepareStatement();
stmt.setInteger(1, inventory.getUid()); // or setLong, depending on the type
stmt.setString(2, inventory.getItemId());
stmt.setString(inventory.getItemData());
int cnt = stmt.executeUpdate();

The PreparedStatement will take care of escaping special SQL characters correctly. They're the best tool agains SQL injection attack, and should always be used when a query has parameters, especially if the parameters come from the end user. See http://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html .

Answer 4

Java strings are stored in UTF-16. So, this code:

String data = new String(this.request.getParameter("data").getBytes("ISO-8859-1"), "UTF-8");

decodes a UTF-16 string (which has been re-encoded from UTF-8 in the HTTP protocol) into a binary array using the ISO-8859-1 charset, and re-encodes the binary array using the UTF-8 charset. This is almost certainly not what you want.

What happens when you use this?

String data = this.request.getParameter("data");
System.out.println(data);

If the second line generates bad data, then your problem is likely in jQuery. Determine that you are indeed getting unicode in your jQuery request:

System.out.println(this.request.getHeader("Content-Encoding"));

If it does not generate bad data, but the data doesn't get stored correctly in mySQL, your problem is at the database level. Make sure your column type supports unicode strings.