读取utf8字符串并将其写入文件

Question

If I have UTF8 encoded string in C (basically a char -or unsigned char ?- array), and I want to write and read it from file (say in binary mode). Is there anything different I need to do with it, as compared to if I were writing/reading just ASCII characters?

Answer 1

Short answer: No, nothing different

Longer Answer: As always, it depends..

It depends on what you're going to use to read the file afterwards. If it's some other application, you may need to give it a hint that the file is UTF-8 encoded text, by sticking a UTF-8 BOM at the front. However, this is typically discouraged, so you can usually revert to the short answer!

However your comments imply your interested in processing the char array, rather than simply reading/writing it. Yes, you may need to do things differently, depending entirely on whay you want to do. Because a single 'unicode character' can be encoded as multiple bytes in the array, the for some operations (counting word lengths in text, for instance) you would need to be aware of the multi-byte characters. But because all the 'extra' bytes in UTF8 have the high bit set, you're never going to get them mixed up with normal characters. So things like string search and replace are typically as per normal ASCII.

Answer 2

If you're just ouputting it (no char counting or modifications), you shouldn't have to worry about it. On Linux with gcc, you can even put UTF8 inside of strings in your source, and it works fine.

Eg:

 puts("řčšéíčšřáčéířáéíščřáéíčřáščéřáěéířěéčšě"); //Will work correctly on Linux

It's just that č , for example, won't be represented by a single char .

Answer 3

As long as you are fine with not actually using the signs for math operations, you should be fine.

UTF8 expects at least 8 bits per character "unit", and C chars, signed or not, are guaranteed to have these. Nothing is different -- except , of course, when you have a habit of adding up "a" to "b" (a nonsense operation on text) or converting to and from integers (which is as okay as it is with "regular" ASCII text with occasional high ASCII characters, ie, if you take care of conversions when they may happen, you should be fine).

With that out of the way: if you are planning to show your output, you might want to use the same type -- signed or unsigned -- as your output library.

If I have to output UTF8 to the screen console (OSX's Terminal window, which is fully capable of showing UTF8) I use regular char strings, so I can use standard stdlib and string functions.