C# JSON Spanish is UTF8 but German is UTF7 to load

Question

I'm building a website that is to have specific languages setup. All pages and their text are in one JSON per language. I'm using the browser Server Variable to tell what language is if we have the JSON conversion for it, we display the text in their language. Default is English if we don't cover it.

However, we load the data with the following code and it works for English and Spanish.

byte[] bytes = File.ReadAllBytes(JSONFilePath);
string someString = Encoding.UTF8.GetString(bytes);
RecordData = (DataSet)JsonConvert.DeserializeObject(someString, (typeof(DataSet)));

However, we noticed it doesn't work for German. We found that we have to use UTF7 for German.

byte[] bytes = File.ReadAllBytes(JSONFilePath);
string someString = Encoding.UTF7.GetString(bytes);
RecordData = (DataSet)JsonConvert.DeserializeObject(someString, (typeof(DataSet)));

If we use UTF8 for German, we get symbols, in some characters. If use UTF7 for Spanish the same thing will happen. We are setting up 10 or so languages, so how do we know which to encode the data with before we deserialize it?

Answer 1

You don't need to encode files for different languages in different code pages. You can, but you will only make your life harder for no good reason. You will need to maintain the mapping between your codepages of choice and your files yourself.

Unicode covers all languages. UTF-8 supports all of Unicode, as do all Unicode encodings.

Make sure you always save all your files as UTF-8, and make sure you always provide an encoding when reading from these files, at which point your code should be:

string someString = File.ReadAllText(JSONFilePath, Encoding.UTF8);
RecordData = (DataSet)JsonConvert.DeserializeObject(someString, (typeof(DataSet)));