Problems encoding/decoding/inserting French accented characters

Question

I am trying to write the following string into a database:

speaker grille moiré effect when printing.

The 'é' character is causing me a massive headache. I cannot use htmlentities , because the database is part of a desktop application. My HTML is UTF-8.

During my (5 days and counting) attempt to debug this problem, I inserted the same string into the database using the desktop application, which seemed to work fine except that the offending character is displayed as '?' -- even using htmlentities or utf8_encode .

These are the error messages I receive on trying to insert the string:

Warning : PDOStatement::execute() [pdostatement.execute]: message: Unclosed quotation mark before the character string 'speaker grille moir'. (severity 15)

Warning : PDOStatement::execute() [pdostatement.execute]: General SQL Server error: Check messages from the SQL Server (severity 15)

Warning : PDOStatement::execute() [pdostatement.execute]: message: Line 1: Incorrect syntax near 'speaker grille moir'. (severity 15)

Please help!

Update (2011-01-18)

Apparently, the database character encoding is SQL_Latin1_General_CP1_CI_AS . Is this the SQL Server equivalent of ISO-8859-1? Isn't ISO-8859-1 PHP's default character encoding?

Update 2 (2011-01-18)

So close! Including charset=UTF-8 in the DSN string allowed me to insert the 'é' character and it shows up correctly in the desktop application. It's still displaying as '?' in the web page though, even though I'm using $text_block = htmlentities( $text_block, ENT_NOQUOTES, 'UTF-8' ); in the preformatting function.

Answer 1

Be sure to pass the dynamic strings correctly using PDO bindings so they get escaped correctly.

Also make sure your database connection is also in UTF-8 charset.

Your collecations in database/fields should also be set to UTF-8, however if they are not this would probably not cause this sort of error.

Answer 2

Have you tried to use in the database the NVARCHAR data type with an accent sensitive collation? Do the following test in your db and tell us if it works:

--speaker grille moiré effect when printing. if object_id('tempdb.dbo.#test') is not null drop TABLE #test create TABLE #test(test NVARCHAR(1000) collate Latin1_General_CI_AS)

insert into #test(test) select 'speaker grille moiré effect when printing.'

select * from #test