将ANSI转换为UTF8

Question

I download a file from the internet in C# Windows Phone 8.1. The problem is, that the downloaded content has strange looking special characters. When I examined the file on my PC with Notepad++, it told me, that the file is encoded in ANSI and I want to read it as UTF8

Here my code

byte[] responseBytes = await client.GetByteArrayAsync("http://somesite/myfile.txt");
string content = Encoding.UTF8.GetString(responseBytes, 0, responseBytes.Length);

But as it is encoded in ANSI, all special characters are displayed strange.

Now after some research, a lot of people have this approach:

Encoding ANSI = Encoding.GetEncoding(1252);
byte[] ansiBytes = ANSI.GetBytes(str);
byte[] utf8Bytes = Encoding.Convert(ANSI, Encoding.UTF8, ansiBytes);
String utf8String = Encoding.UTF8.GetString(utf8Bytes);

but in WP 8.1, the routine Encoding.GetEncoding(1252) is invalid, as well as Encoding.Default . What can I do, to have my string in UTF8?

Answer 1

In general (but apparently not on Windows Phone), the way to do this is to simply use the correct encoding from the get-go:

string content = Encoding.Default.GetString(responseBytes, 0, responseBytes.Length);

Where Encoding.Default is defined as:

an encoding for the operating system's current ANSI code page.

… What you are currently attempting to do is interpret the bytes in an incorrect encoding, and then try to re-encode them. This won't generally work.

---

But as you've said, Windows Phone does not support this . So what you do instead is to manually create a byte-to-character translation table for Windows-1252 and look up the characters. You can then either manually loop over the input buffer, or, for extra points, create a new class which derives from System.Text.Encoding and which implements the required encoding.

In fact, if I remember C# conversions correctly, you don't even need a lookup table. The following is a rudimentary but (for your purposes) sufficient Encoding implementation:

class Windows1252Encoding : System.Text.Encoding {
    public override int GetByteCount(char[] chars, int index, int count) {
        return count;
    }

    public override int GetBytes(char[] chars, int charIndex, int charCount, byte[] bytes, int byteIndex) {
        Array.Copy(chars, charIndex, bytes, byteIndex, charCount);
        return charCount;
    }

    public override int GetCharCount(byte[] bytes, int index, int count) {
        return count;
    }

    public override int GetChars(byte[] bytes, int byteIndex, int byteCount, char[] chars, int charIndex) {
        Array.Copy(bytes, byteIndex, chars, charIndex, byteCount);
        return byteCount;
    }

    public override int GetMaxByteCount(int charCount) {
        return charCount;
    }

    public override int GetMaxCharCount(int byteCount) {
        return byteCount;
    }
}

This seems to work, but I cannot test it on Windows Phone, only on Mono.