Not about to deserialize json kept as string in Python using json.load

Question

strdata = strdata + json.dumps(data, default=lambda o: o.__dict__)

I'm using this to concatenate json data from various api calls, into a string.

Now, when I want to read the data/load the variable "strdata" into a json format, using

json.loads(strdata)

but it doesn't work. I presume that considering that I'm concatenating a serialized string with another, I should dump the entire strdata first and then load it again. But that doesn't work either.

Answer 1

Initialize strdata as a list:

strdata = []

Inside the loop, append the JSON dumps to the list:

strdata.append(json.dumps(data, default=lambda o: o.__dict__))

To store the list in a file:

json.dump(strdata, f)

To load the list from the file:

strdata = json.load(f)

To retrieve the original data (or the data.__dict__ proxy), call json.loads on each item in strdata :

[json.loads(item) for item in strdata]

---

JSON has a well-defined format . You can not create valid JSON (such as a list of elements) by simply concatenating two JSON strings.

---

There is a more sophisticated way to handle this problem: use a custom encoder ( cls=CustomEncoder ), and a custom decoder ( object_hook=custom_decoder ):

import json

class Foo(object):
    def __init__(self, x=1, y='bar'):
        self.x = x
        self.y = y

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Foo):
            return obj.__dict__
        else:
            return json.JSONEncoder.default(self, obj)

filename = '/tmp/test.json'
with open(filename, 'w') as f:
    json.dump(
        [Foo(1, 'manchego'), Foo(2, 'stilton'), [{'brie': Foo(3,'gruyere')}]],
        f, cls=CustomEncoder)

def custom_decoder(dct):
    try:
        return Foo(**dct)
    except TypeError:
        return dct

with open(filename, 'r') as f:
    newfoo = json.load(f, object_hook=custom_decoder)
print(newfoo)
# [{"y": "manchego", "x": 1}, {"y": "stilton", "x": 2}, [{"brie": {"y": "gruyere", "x": 3}}]]

The advantage of doing it this way is that it requires only one call to json.dump to store the data, and only one call to json.load to retrieve the data.