Importing utf8 characters to MYSQL

Question

I'm trying to import a CSV into a MYSQL database. My CSV contains utf8 characters like â when I import these to my database they are replaced with ? marks.

I have tried the following:

a) Importing directly through PHPMYADMIN b) Importing via Navicat (importing as UTF8) c) Changing the encoding to UTF8 with open office / mac excel 2010 (seems to make them question marks as well) d) Saving as CSV ms-dos (not sure if it would make a difference or not)

Can anyone shed some light on what I could be doing wrong? Is it something to do with my Collation?

Answer 1

A few things to check:

1) Make sure your collation for the table and/or rows is utf8_general_ci (for case insensitive) or utf8_bin (for binary exact). There are many others but in my experience one of these two is most common.

2) Make sure your php code files are encoded as UTF-8 Without Byte Order Mark (BOM). This prevents PHP from losing the encoding in-code.

3) If you're using the multibyte functions in php, ensure you have default_charset = "utf-8" set in php.ini

4) Ensure your database connection (probably a setting in phpmyadmin, not certain as I don't use it) is calling mysql_set_charset('utf8', $connection); just after you open the connection, and before sending queries.

Answer 2

Yes it is to do with your collation. You need to change it to a utf8_* collation.

Answer 3

Declaring the character set in the import statement worked for me:

mysql> LOAD DATA INFILE '/usr/share/mysql/file.csv' 
       INTO TABLE Table1
       CHARACTER SET utf8;