How to remove double quotes that result in illegal json in json？

Question

I have received a json from server, which is not legal, because it contains some unexpected double quotes.

The json is like the following:

{
    "result": {
        "books": [{
            "book_name": "All Around Weekly 04 - "Phantom," the Prairie Trapper"
        }]
    }
}

I have deleted the unrelated parts of this json and kept the most concerned part.

Note this line：

 "book_name": "All Around Weekly 04 - "Phantom," the Prairie Trapper"

The value of book_name is a String ，but All Around Weekly 04 - "Phantom," the Prairie Trapper have two unwanted double qoutes in it, which are not escaped.

I want the server side to do some escaping, so I can fetch the legal json.

Actually, the server side wants me to solve this problem, which looks easy at first glance.

However, I have not found a way to find out the targeted double quotes, which result in illegal json.

One way I have tried is:

public static String toJsonString(String s) {
    char[] tempArr = s.toCharArray();
    int tempLength = tempArr.length;
    for (int i = 0; i < tempLength; i++) {
        if (tempArr[i] == ':' && tempArr[i + 1] == '"') {
            for (int j = i + 2; j < tempLength; j++) {
                if (tempArr[j] == '"') {
                    if (tempArr[j + 1] != ',' && tempArr[j + 1] != '}') {
                        tempArr[j] = '\''; // replace double quotes with single quote.
                    } else if (tempArr[j + 1] == ',' || tempArr[j + 1] == '}') {
                        break;
                    }
                }
            }
        }
    }
    return new String(tempArr);
}

It doesn't work for this kind of illegal json:

{
    "books": ["kotlin books", "java "books ""]
}

I feel worried about that though I have not come across this kind of json.

Answer 1

I have no way to prove this mathematically, but I strongly believe it is impossible. For every kind of heuristic you come up with, I am pretty certain you will also be able to come up with a counter-example that is ambiguous.

In short: you need to get back to the owner of the service and tell them that they either need to give you a complete, unambiguous specification of the syntax of their home-grown almost-but-not-quite-JSON data format so that you can implement a parser for that data format, or use an already existing data format with an already existing complete, unambiguous specification … such as JSON .

Just as an example of how easy it is to get things wrong: JSON itself was created with the intention of being a strict subset of JavaScript / ECMAScript, in particular, a strict subset of JavaScript / ECMAScript Object Literal Syntax. (Meaning that every valid JSON Document should at the same time also be a valid JavaScript / ECMAScript program, in particular, a valid JavaScript / ECMAScript Object Literal.) And the JSON specification was written by someone who had both extensive experience with writing specifications (and also with writing parsers) as well as extensive experience with JavaScript / ECMAScript.

And yet , it was discovered later that due to a tiny oversight in the JSON specification, it was actually possible to have valid JSON documents that are illegal ECMAScript programs. Because JSON deliberately does not contain versions so that it can never be changed, the ECMA Technical Committee 39 responsible for ECMAScript actually decided to change ECMAScript so that it becomes a strict superset of JSON in ECMAScript 2019.