Converting JSON-like object to JSON

Question

I have a JSON file that was filtered and lost the original structure. Each line of the file looks like this:

{u'spec1': {u'property1': u'12345', u'property2': 1234}, u'spec2': {u'property3': u'98754', u'property4': u'val1'}}

json.loads doesn't work and ast.literal_eval doesn't work. I guess I have 2 issues: remove unicode and change ' to ". Can anyone provide some pointers on where to start?

Answer 1

Assuming you are using Python 2.X.

json.loads takes a str or unicode as its param. The string you give is not a valid json string. So we should do some pre-cooking work.

import re, json

json_str = """{u'spec1': {u'property1': u'12345', u'property2': 1234}, u'spec2': {u'property3': u'98754', u'property4': u'val1'}}"""
json_str = json_str.replace("\'", "\"")
json_str = re.sub(r"u\"", "\"", json_str)

json_dict = json.loads(json_str)

Then the json_dict will be a dictionary inflated from your json string.

Answer 2

Looks like this works (in Python3):

$ python3.4
Python 3.4.0 (default, Jun 19 2015, 14:20:21) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> j = """{u'spec1': {u'property1': u'12345', u'property2': 1234}, u'spec2': {u'property3': u'98754', u'property4': u'val1'}}"""
>>> json.loads(j.replace("'","\"").replace('u"','"'))
{'spec1': {'property1': '12345', 'property2': 1234}, 'spec2': {'property4': 'val1', 'property3': '98754'}}

As you can see, I've replaced both ' to " chars, and (thus came) u" to " patterns.

Hope this helps.

a.

Answer 3

Here is my take (Python 2.7).

import StringIO
import ast
file = u"""{u'spec1': {u'property1': u'12345', u'property2': 1234}, u'spec2': {u'property3': u'98754', u'property4': u'val1'}}
{u'spec2': {u'property1': u'12345', u'property2': 1234}, u'spec3': {u'property3': u'98754', u'property4': u'val1'}}
{u'spec4': {u'property1': u'12345', u'property2': 1234}, u'spec2': {u'property5': u'98754', u'property4': u'val1'}}
{u'spec6': {u'property1': u'12345', u'property2': 1234}, u'spec2': {u'property7': u'98754', u'property4': u'val1'}}
"""
buffer = StringIO.StringIO(file)
lines = buffer.readlines()
dicts = []
for line in lines:
    dicts.append(ast.literal_eval(line))
print dicts

Don't look at StringIO , it's there to emulate file-reading. What I'm proposing is to read the file by line and do literal_eval by line.

For me it was the only way to make it work without errors.