在Java中错误地修复了ISO-8859-1解码的UTF-8字符串

Question

I have to deal with a library which is not in my control. It delivers a string which it decoded from a byte stream with ISO-8859-1. However the byte stream is UTF-8. So obviously the resulting string I get is wrong if it contains non ASCII characters.

So what I do to fix this is to convert the string back to the byte stream and decode it again with UTF-8. Like this:

byte[] raw = inputText.getBytes(StandardCharsets.ISO_8859_1);
String correctedText = new String(raw, StandardCharsets.UTF_8);

I tested it with many examples and it seems to work. Is this always correct however or are there cases where this would not work? In other words: are there cases where decoding / reencoding any arbitrary byte array with ISO-8859-1 would not result in the original byte array?

Answer 1

Since ISO-8859-1 is a 1 byte per character encoding, it will always work. The UTF-8 bytes are converted to incorrect characters, but luckily there's no information lost.

Changing the characters back to bytes using ISO-8859-1 encoding gives you the original byte array, containing characters encoded in UTF-8 , so you can then safely reinterpret it with the correct encoding.

The opposite of this is not (always¹) true, as UTF-8 is a multibyte encoding. The encoding process may encounter invalid byte sequences and replace them with the replacement character ? . At that point you've lost information and can't get the original bytes back anymore.

¹ If you stick to characters in the 0-127 range it will work, as they're encoded in UTF-8 using a single byte.

Answer 2

UTF-8 and ISO-88-1 encode ASCII charactesrs at the same way. Given this You should not have any losses only since your original input is ASCII.