如何转换代理对字符？

Question

I have a web service that takes an XML String as input and it is primarily in UTF-8. However, there is a possibility that surrogate pairs can get mixed into the string and those particular characters become unreadable when it is being processed by my application.

I am reading in an xml file like so (I have a feeling this part messes things up):

String xmlFile = new String(Files.readAllBytes(Paths.get("test.xml")),"UTF-8");

I know that I can detect this with when I loop through every character in the string:

Character.isSurrogatePair(high, low)

What I want to know is if there is a way to convert a surrogate pair to something that can be recognized properly in UTF-8. For example "" is recognizable in UTF-8 since it has 3 bytes but "𠃮" has 4 bytes (surrogate pair) but the graphical display is identical.

Answer 1

Your code is 100% fine (if the encoding is indeed UTF-8). Surrogate pairs is a way UTF-16 encode a Unicode code point as two char s (2x2 bytes). That is covered by UTF-8 as a longer multibyte sequence (upto 6 bytes, in 2017).

Unicode itself just numbers code points (symbols). Those numbers are then encoded with UTF- nn so no errors can happen such as searching the byte for / and finding it falsely. UTF-8 uses high bits, and UTf-16 does a similar trick, with "surrogate pairs." Unicode & UTF is a solid design.

Now Unicode did grow over time, and the standards expanded likewise.

So running with java 6 you might not have the same Unicode power (range) as later versions. Likewise old non-java programs and fonts might have their white spots.

Most likely something in the data is fishy. Reading byte blocks and converting every block to a String would cause invalid characters at block boundaries.