Special characters are stored as ? in my mysql DB

Question

I use the following code to store and update data in my db.

<?php
header("Content-Type: text/html;charset=UTF-8", true);
mysql_connect("localhost", "test", "test") or die(mysql_error());
    mysql_select_db("test") or die(mysql_error());

    // mysql_set_charset('UTF8');
    mysql_query("SET NAMES 'utf8'");
    mysql_query("SET CHARACTER SET utf8");
    mysql_query("SET COLLATION_CONNECTION = 'utf8_unicode_ci'");




    mysql_query("UPDATE `test`.`english` SET age = '".$age."', city = '".mysql_real_escape_string($_REQUEST['city'])."', aboutMe = '".$_REQUEST['about']."' WHERE `english`.`username` = '".$_REQUEST['username']."' LIMIT 1 ;") 
    or die(mysql_error());  

The code is working well. But if the string, eg "city" contains a ä,ö,ü it is stored as ? So for example the word "Bär" goes to "B?r".

I have already added these utf8 things but it does not help. The server is only supporting php4.x

Edit: The DB is using latin1_general_ci as collation.

Answer 1

Use utf8 as collation, maybe utf8_bin or utf8_unicode_ci

(The ci means it's case insensitive so comparsions like "John" = "JOHN" return true). You can do this by running the following query in MySQL:

alter table `test`.`english` convert to character set utf8 collate utf8_unicode_ci;

Answer 2

change the table collation to utf8_bin .

To change the default character set and collation of a table including those of existing columns (note the convert to clause):

ALTER TABLE tableName CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin

in utf8_general_ci

ä = a
ö = o
ü = u

but in utf8_bin

ä != a
ö != o
ü != u

Answer 3

Keep ut8_unicode_ci and add this to your php code:

mb_language('uni');
mb_internal_encoding('UTF-8');
mysql_query("set names 'utf8'",$connection); 

I am using this everywhere (Cyrillic, Chinese etc ) and it is working for every query (insert or update).

 <?php
        header("Content-Type: text/html;charset=UTF-8", true);
        $connection = mysql_connect("localhost", "test", "test") or die(mysql_error());

        mb_language('uni');
        mb_internal_encoding('UTF-8');
        mysql_query("set names 'utf8'",$connection);

        mysql_select_db("test") or die(mysql_error());
        mysql_query("UPDATE `test`.`english` SET age = '".$age."', city = '".mysql_real_escape_string($_REQUEST['city'])."', aboutMe = '".$_REQUEST['about']."' WHERE `english`.`username` = '".$_REQUEST['username']."' LIMIT 1 ;") 
        or die(mysql_error());  

Answer 4

I got it working. I have no explanation for this behavior. I have changed in the php file from utf8 to latin1:

mysql_query("SET NAMES 'latin1'");
mysql_query("SET CHARACTER SET latin1");
mysql_query("SET COLLATION_CONNECTION = 'latin1_general_ci'");

In the MySQL DB the collation is utf8_unicode_ci for the table and column.

It is working now for all characters beside the € symbol.