Rails / MySQL奇怪的UTF-8编码问题

Question

I have an app that uses Cyrillic (Macedonian) alphabet. I have an alphabet menu with all of the letters (manually typed as an array which I'm calling it from a helper), that links to items where the first character is that letter. So, it seems that "К" and "Ќ" (also "Г" and "Ѓ") are listing the same items, as if they were using the same letter. Works great in development, not sure why it does this on production. I have set UTF-8 encoding on production database.

Here's the prod log. It's not getting the same character.

App 18197 stderr: Started GET "/letterfilter?title=%D0%8C" for IP at2015-07-30 12:03:46 -0400
App 18197 stderr: Processing by PostsController#letterfilter as HTML
App 18197 stderr:   Parameters: {"title"=>"Ќ"}
App 18197 stderr:   Rendered posts/letterfilter.html.haml within layouts/application (4.3ms)
App 18197 stderr:   Rendered posts/_search.html.haml (0.8ms)
App 18197 stderr:   Rendered shared/_header.html.haml (9.6ms)
App 18197 stderr:   Rendered shared/_footer.html.haml (0.2ms)
App 18197 stderr: Completed 200 OK in 18ms (Views: 16.6ms | ActiveRecord: 0.2ms)

App 18197 stderr: Started GET "/letterfilter?title=%D0%9A" for IP at 2015-07-30 12:03:51 -0400
App 18197 stderr: Processing by PostsController#letterfilter as HTML
App 18197 stderr:   Parameters: {"title"=>"К"}
App 18197 stderr:   Rendered posts/letterfilter.html.haml within layouts/application (4.9ms)
App 18197 stderr:   Rendered posts/_search.html.haml (0.7ms)
App 18197 stderr:   Rendered shared/_header.html.haml (7.7ms)
App 18197 stderr:   Rendered shared/_footer.html.haml (0.2ms)
App 18197 stderr: Completed 200 OK in 17ms (Views: 14.2ms | ActiveRecord: 0.9ms)

What might be causing this issue? Should I update my database encoding to utf8mb4?

Any help is welcomed. Thanks.

Answer 1

Ќ is hex D08C in utf8 or utf8mb4. Cyrillic is completely covered by either CHARACTER SET . К is D09A , as can be seen in the ?title=

Hmmm, this is quite interesting:

mysql> SELECT 'К' = 'Ќ' COLLATE utf8_bin AS bin,
              'К' = 'Ќ' COLLATE utf8_general_ci AS general,
              'К' = 'Ќ' COLLATE utf8_unicode_ci AS unicode;
+-----+---------+---------+
| bin | general | unicode |
+-----+---------+---------+
|   0 |       1 |       0 |
+-----+---------+---------+

mysql> SELECT 'Г' = 'Ѓ' COLLATE utf8_bin AS bin,
              'Г' = 'Ѓ' COLLATE utf8_general_ci AS general,
              'Г' = 'Ѓ' COLLATE utf8_unicode_ci AS unicode;
+-----+---------+---------+
| bin | general | unicode |
+-----+---------+---------+
|   0 |       1 |       0 |
+-----+---------+---------+

The bit patterns are different, so utf8_bin collates unequal. But usually whenever general is equal, so is unicode.

Back to your question... What do you mean "it's not getting the same character"?