通过格式不正确的JSON文件python进行解析

Question

I was given a dataset that I need to do some data analytics on it. Each dataset that was given to me is in a json file. The problem that I am having is that I noticed that each json object is not separated by a ',' so I can't just do a simple json dump into a variable. And I cannot just add in a ',' in between each object because each file has over 100 json objects and there is about 100 files, so it would take a long time to do that. So I was wondering what I could do to fix this issue that I am having.

Answer 1

Since you aren't providing an example of your data, you could extend your JSONDecoder adding a class like:

import json

class ComplexEncoder(json.JSONDecoder):
    def decode(self, obj):
        obj = obj.replace(" ", ", ")
        print(obj)
        return json.JSONDecoder.decode(self, obj)

a = json.loads('{"a":1 "b":2}', cls=ComplexEncoder)
print(a)
# {'a': 1, 'b': 2}

Basically just replace that space with a comma, if you have spaces between the : and the value, make a regex that don't replace that.

I think you're refering to json.loads() instead of json.dumps

Answer 2

You could try using littletable , which will import files containing consecutive, undelimited (even multiline) JSON objects.

import littletable as lt

data = """
{"a": 100, "b": 200, "c": 300}
{"a": 101, "b": 201, "c": 301}
{
    "a": 102, 
    "b": 202, 
    "c": 302

}
"""

json_table = lt.Table()
# for this post we import from the data using a Python string;
# in your program, just do json_table.json_import('data_file.json')
json_table.json_import(data)
for row in json_table:
    print(row.a, row.b, row.c)

Prints:

100 200 300
101 201 301
102 202 302

Once it is imported, you could re-export it as a CSV, or just use the table like a normal Python list and serialize it any way you like.

Disclosure: I am the author of littletable