需要说明使用C ++在Linux上创建utf-8编码的文件

Question

I need some explanations on encodage of files using g++ on Linux.

I have an easy code :

int main ()
{
  FILE * pFile;
  char buffer[] = { 'x' , 'y' , 'z' ,'é' };
  pFile = fopen ("myfile", "wt, ccs=UTF-8");
  //pFile = fopen ("myfile", "wt");
  fwrite (buffer , sizeof(char), sizeof(buffer), pFile);
  fclose (pFile);
  return 0;
}

Even if the "ccs=UTF-8" part is added on the fopen line, the output file of this program is always encoded in iso-8859-1. However, if I create a file using vi on Linux containing theses charaters, the resulting file is UTF-8 encoded (I use the command "file myfile" to see the encoding mode of the file, and a "xxd -b myfile" confirm this behavior).

So I would like to undestand :

1- Why g++ on Linux doesn't create a UTF-8 file by default?

2- What is the aim of the ccs=UTF-8 if the file created is not encoded in UTF-8?

3- How can I create an UTF-8 file based on this simple code?

Thanks.

Answer 1

Your file may appear to be in ISO-8859-1, but it's actually not. It's simply broken.

Your file contains byte A9 , which is the lower byte of UTF-8 representation of é .

When you wrote 'é' , the compiler should have warned you:

 aaa.c:4:38: warning: multi-character character constant [-Wmultichar]
     char buffer[] = { 'x' , 'y' , 'z' ,'é' };
                                         ^

char is not a type for a character, it's a type for one byte. GCC treats multibyte character literals as big-endian integers. Here, you cast it immediately to char , leaving the lowest byte: A9

(BTW, é in ISO-8859-1 is E9 , not A9 )

You open your file with an encoding, but then you save bytes into it. The bytes correspond to ISO-8859-1 characters xyz© .

If you want to write characters, not bytes, then use wchar_t instead of char and fputws instead of fwrite

#include <stdio.h>
#include <wchar.h>

int main ()
{
  FILE * pFile;
  // note final zero and L indicating wchar_t literal
  wchar_t buffer[] = { 'x' , 'y' , 'z' , L'é' , 0};
  // note no space before ccs
  pFile = fopen ("myfile", "wt,ccs=UTF-8");
  fputws(buffer, pFile);
  fclose (pFile);
  return 0;
}