Dealing with Unicode characters in an ASCII file?

Question

I have an XML file which I saved as ASCII/UTF-8 using XmlSerializer in C#. One field contains a folder path location. I have recently discovered that on non-English language Windows systems, there can be special characters in the path field. I could save the entire file as Unicode/UTF-16 but that doubles the file size for the sake of a few characters.

Is there a way to insert non-ASCII characters into an otherwise ASCII string?

Answer 1

There's no such thing as ASCII/UTF-8. Those are two distinct encodings, that in fact encode different character sets. I suspect that you are actually using ASCII, or perhaps Windows ANSI, at present.

UTF-8 is a complete encoding for Unicode. If the file only contains ASCII characters then the UTF-8 encoding is identical to the ASCII encoding. And if your files are predominantly English, then UTF-8 is the Unicode encoding that produces the smallest files.

Conclusion: use UTF-8.