Incorrect string value for Cangjie character using MySQL utf8 collation (4 byte support)

Question

I have an application that is saving user input to a table in my database. The database was originally set to utf8 (MySQL v5.7) but from reading it was suggested that MySQL supports only 3 bytes with their utf8 collation and that an upgrade to utf8mb4 was needed for 4 byte support.

I'm currently running some testing by saving text in Cangjie and for the most part it seems fine, however when trying to save the following character '𤍢', I receive the following error:

'Incorrect string value: \'\\xF0\\xA4\\x8D\\xA2\\xE5\\x8F...\' for column \'content\' at row 1'

I upgraded to utf8mb4 on the database, table and column level but still saw the error.

I tried manually inserting the content at the DB level as well (rather than through the application) and got the same error, so I know it's not an implementation problem.

Can anyone suggest a reason why this might be the case? I thought utf8mb4 would have covered this

Answer 1

SOLVED: I used the following set of commands (modified to meet my needs) to update the default collation: Source: https://mathiasbynens.be/notes/mysql-utf8mb4

# For each database:
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = 
utf8mb4_unicode_ci;
# For each table:
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE 
utf8mb4_unicode_ci;
# For each column:
ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) 
CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
# (Don’t blindly copy-paste this! The exact statement depends on the 
column type, maximum length, and other properties. The above line is 
just an example for a `VARCHAR` column.)

This alone did not work, I also had to do this in order to update a number of global variables such as character_set_client, character_set_connection etc:

set names utf8mb4 collate utf8mb4_unicode_ci;

This was just for the connection I had open to the database. The client-side code actually works after the initial charset update.