how to remove double quotes out side of generated json from string list using python

Question

My JSON is imput json { 'debug': None, 'traceback': None, 'rsname': None, 'resolutionStatus': 0, 'algorithmResponseInfo': None, 'rules': "['check']", 'rulesResults': '-RNCID Found', 'mmlCommand': 'tk_com', 'KMmmlCommand': 'cat "tk"', 'similarityScore': {'similarityUI': 100 }} which is invalid and I want to convert it into valid json I want to remove "" double quotes and single backslash from rules column (list) using python. I have already tried the following but it didn't work -

data = data.replace('"',"\\'").replace("'", '"').replace("True","true").\
             replace("None","null").replace("False","false").replace("\'", '"').replace("'", '"')

data = json.loads(data)

output is - data = { "debug": null, "traceback": null, "rsname": null, "resolutionStatus": 0, "algorithmResponseInfo": null, "rules": \ "["check"]", "rulesResults": "-RNCID Found", "mmlCommand": "tk_com", "KMmmlCommand": "cat "tk"", "similarityScore": { "similarityUI": 100 }}

but i need to remove extra backshlash and double quotes expected is - data = { "debug": null, "traceback": null, "rsname": null, "resolutionStatus": 0, "algorithmResponseInfo": null, "rules": ["check"], "rulesResults": "-RNCID Found", "mmlCommand": "tk_com", "KMmmlCommand": "cat "tk"", "similarityScore": { "similarityUI": 100 }}

Answer 1

What you have there is not JSON at all, but a Python literal. It's just that Python's literals are very similar to JSON. The safe way to convert this is using ast.literal_eval:

from ast import literal_eval
import json
data = (
    "{ 'debug': None, 'traceback': None, 'rsname': None,"
    " 'resolutionStatus': 0, 'algorithmResponseInfo': None, "
    "'rules': \"['check']\", 'rulesResults': '-RNCID Found', "
    "'mmlCommand': 'tk_com', 'KMmmlCommand': 'cat \"tk\"',"
    " 'similarityScore': {'similarityUI': 100 }}"
python_doc = literal_eval(data)
json_str = json.dumps(python_doc)

# {"debug": null, "traceback": null, "rsname": null, "resolutionStatus": 0,
# "algorithmResponseInfo": null, "rules": "['check']",
#  "rulesResults": "-RNCID Found", "mmlCommand": "tk_com",
#  "KMmmlCommand": "cat \"tk\"", "similarityScore": {"similarityUI": 100}}
print(json_str)

The unsafe alternative to literal_eval is eval . The problem with eval is that if an attacker can control the input, they can cause arbitrary code to run.