单个UTF-8字符到字节

Question

If I am converting a UTF-8 char to byte, will there ever be a difference in the result of these 3 implementations based on locale, environment, etc.?

byte a = "1".getBytes()[0];
byte b = "1".getBytes(Charset.forName("UTF-8"))[0];
byte c = '1';

Answer 1

Your first line is dependent on the environment, because it will encode the string using the default character encoding of your system, which may or may not be UTF-8 .

Your second line will always produce the same result, no matter what the locale or the default character encoding of your system is. It will always use UTF-8 to encode the string.

Note that UTF-8 is a variable-length character encoding. Only the first 127 characters are encoded in one byte; all other characters will take up between 2 and 6 bytes.

Your third line casts a char to an int . This will result in the int containing the UTF-16 character code of the character, since Java char stores characters using UTF-16. Since UTF-16 partially encodes characters in the same way as UTF-8, the result will be the same as the second line, but this is not true in general for any character.

Answer 2

In principle the question is already answered, but I cannot resist to post a little scribble, for those who like to play around with code:

import java.nio.charset.Charset;

public class EncodingTest {

    private static void checkCharacterConversion(String c) {
        byte asUtf8 = c.getBytes(Charset.forName("UTF-8"))[0];
        byte asDefaultEncoding = c.getBytes()[0];
        byte directConversion = (byte)c.charAt(0);
        if (asUtf8 != asDefaultEncoding) {
            System.out.println(String.format(
                "First char of %s has different result in UTF-8 %d and default encoding %d",
                c, asUtf8, asDefaultEncoding));
        }
        if (asUtf8 != directConversion) {
            System.out.println(String.format(
                "First char of %s has different result in UTF-8 %d and direct as byte %d",
                c, asUtf8, directConversion));
        }
    }

    public static void main(String[] argv) {

       // btw: first time I ever wrote a for loop with a char - feels weird to me
       for (char c = '\0'; c <= '\u007f'; c++) {
           String cc = new String(new char[] {c});
           checkCharacterConversion(cc);
       }
    }
}

If you run this eg with:

java -Dfile.encoding="UTF-16LE"  EncodingTest

you will get no output. But of course every single byte (ok, except for the first) will be wrong if you try:

java -Dfile.encoding="UTF-16BE"  EncodingTest

because in "big endian" the first byte is always zero for ascii chars. That is because in UTF-16 an ascii character '\\u00xy is represented by two bytes, in UTF16-LE as [xy, 0] and in UTF16-BE as [0, xy]

However only the first statement produces any output, so b and c are indeed the same for the first 127 ascii characters - because in UTF-8 they are encoded by a single byte. This will not be true for any further characters, however; they all have multi-byte representations in UTF-8.