How to get a UTF-8 JSON

Question

I'm using LitJSON library but things gets a little bit odd.

Do you know any JSON library that keeps the accents when converting ?

Here's the test :

test.json

[{"id":"CS_001","name":"L'élément","type":"Tôt"},{"id":"CS_002","name":"L'outrage","type":"Tôt"},{"id":"CS_003","name":"Test","type":"Tôt"}]

test.cs

public class test : MonoBehaviour {
    private string jsonString;
    private JsonData cardData;
    JsonData database;

    void Start () {
        jsonString = File.ReadAllText (Application.dataPath + "/test.json");
        cardData = JsonMapper.ToObject (jsonString);
        database = JsonMapper.ToJson (cardData);
        Debug.Log (database.ToString ());
    }
}

And the Debug.Log turns to :

[{"id":"CS_001","name":"L'\u00E9l\u00E9ment","type":"T\u00F4t"},{"id":"CS_002","name":"L'outrage","type":"T\u00F4t"},{"id":"CS_003","name":"Test","type":"T\u00F4t"}]

Any idea how to get a proper Json ? Even if it's with another JSON library.

Thank you very much.

Answer 1

Content-type: application/json; charset=utf-8 designates the content to be in JSON format, encoded in the UTF-8 character encoding. The default encoding for JSON is UTF-8. In this case the receiving server apparently does not know that it's dealing with JSON in the UTF-8 encoding and you may need to convert it manually:

byte[] encodedBytes = Encoding.UTF8.GetBytes(jsonString);
Encoding.Convert(Encoding.UTF8, Encoding.Unicode, encodedBytes);

or just try to specify the content type on your request:

content-type: application/json; charset=utf-8

Answer 2

Here's an example using Json.Net to deserialize the string:

using System;
using Newtonsoft.Json;
using System.Collections.Generic;

public class Program
{
    public static void Main()
    {
        // Deserialize the JSON into a list of CardData
        var ob = JsonConvert.DeserializeObject<List<CardData>>("[{\"id\":\"CS_001\",\"name\":\"L'élément\",\"type\":\"Tôt\"},{\"id\":\"CS_002\",\"name\":\"L'outrage\",\"type\":\"Tôt\"},{\"id\":\"CS_003\",\"name\":\"Test\",\"type\":\"Tôt\"}]" );

        /*
          The output will be:
            id: CS_001, name: L'élément, type: Tôt
            id: CS_002, name: L'outrage, type: Tôt
            id: CS_003, name: Test, type: Tôt
        */
        foreach(var i in ob){
            Console.WriteLine(i);  
        }
    }
}

// Class that will hold the deserialized data
// For demo puposes
public class CardData
{
    public string id { get; set; }
    public string name { get; set; }
    public string type { get; set; }

    public override string ToString(){
        return String.Format("id: {0}, name: {1}, type: {2}",id, name, type);   
    }
}

Live demo available here

Answer 3

It is possible that you read the text file using the wrong encoding. Try to use the overload for File.ReadAllText that takes in an Encoding argument and pass it UTF8.