在 python json 中的对象内部序列化 object 引用

Question

I am attempting to saving my QGraphicsScene, where the items being drawn may contain references to other items on the scene.

This raises a major concern as when saving to json, and loading from it, the object may appear at correct position but the references is something I am unsure of how to implement.

Is there a way how I can store which object a reference is to? (the referred object is also stored in the json)

I use getstate and setstate methods to serialize my objects, and the ones that will never have references but are always referred are added to the json first so the reference objects are there when loaded.

Here is an MRE:

import json
class shape():
    def __init__(self, x, y, width, height):
        self.x = x
        self.y = y
        self.width = width
        self.height = height

    def __getstate__(self):
        return {
            "x": self.x,
            "y": self.y,
            "width": self.width,
            "height": self.height
        }

    def __setstate__(self, dict):
        pass

class line():
    def __init__(self, shape1=None, shape2=None):
        self.shape1 = shape1
        self.shape2 = shape2

    def __getstate__(self):
        return {
            "shape1": self.shape1,
            "shape2": self.shape2
        }

    def __setstate__(self, dict):
        self.shape1 = dict['shape1']
        self.shape2 = dict['shape2']

if __name__ == "__main__":
    shapes = [shape(4, 4 , 4, 4) for _ in range(4)]
    line1 = line(shapes[0], shapes[1])
    line2 = line(shapes[2], shapes[3])
    items = {
        "shapes": [item.__getstate__() for item in shapes],
        "lines": [line1.__getstate__(), line2.__getstate__()] 
    }
    with open("test.json", "w") as file:
        json.dump(items, file)
    with open("test.json", "r") as file:
        item = json.load(file)

From the above example, i want to retain the line1 and line2 to have references to the specific shapes even after loading.

Answer 1

You probably need a JSONEncoder of sorts. Something that can parse the input data to json.dump before it gets passed through the sanitizers of the json lib.

You can do this by hooking in a cls=Typer class like so:

from json import JSONEncoder, dumps, loads
from datetime import date, datetime

class JSON_Encoder:
    def _encode(obj):
        if isinstance(obj, dict):
            ## We'll need to iterate not just the value that default() usually gets passed
            ## But also iterate manually over each key: value pair in order to trap the keys.

            for key, val in list(obj.items()):
                if isinstance(val, dict):
                    val = loads(dumps(val, cls=JSON_Typer)) # This, is a EXTREMELY ugly hack..
                                                            # But it's the only quick way I can think of to 
                                                            # trigger a encoding of sub-dictionaries. (I'm also very tired, yolo!)
                else:
                    val = JSON_Encoder._encode(val)
                del(obj[key])
                obj[JSON_Encoder._encode(key)] = val
            return obj
        elif hasattr(obj, 'json'):
            return obj.json()
        elif isinstance(obj, (datetime, date)):
            return obj.isoformat()
        elif isinstance(obj, (list, set, tuple)):
            r = []
            for item in obj:
                r.append(loads(dumps(item, cls=JSON_Typer)))
            return r
        else:
            return obj

class JSON_Typer(JSONEncoder):
    def _encode(self, obj):
        return JSON_Encoder._encode(obj)

    def encode(self, obj):
        return super(JSON_Typer, self).encode(self._encode(obj))

This code is by far not perfect , I'm aware of many issues with it. But it gets the job done 99% of the time as long as you're careful with circular dependencies.

With the above code, all you need to do is two things. The first being implementing a json function as the JSON_Encoder will be looking for elif hasattr(obj, 'json'): (you could change this to to_json() or something if you feel that's more logical) .

Secondly, when doing json.dump(items, file) , simply change it to:

json.dump(items, file, cls=JSON_Typer)

As it will append the pre-parser for all the objects seen in the JSON structure. Note here that it will traverse the dictionary, so it might feel a bit slow if your JSON structure is massive. And you'll need to implement a logic to convert yout shape into a representation of it so you upon json.load can interpret the data and do a replacement of that position in the JSON.

This is the best way I've fond over the years, it's a bit hacky but I've had some success with it.

Answer 2

While @Torxed answers serializability of python classes, My main objective was saving references to objects. But since at loading I was creating new objects, storing references in the json file became a challenge.

My approach was using hex(id(ref)) to assign id to the object, and the same ref id gets stored in the referring object.

shapeDict = {}
class shape():
    ...

    def __getstate__(self):
        return {
            "x": self.x,
            "y": self.y,
            "width": self.width,
            "height": self.height,
            "id": hex(id(self))
        }

    def __setstate__(self, dict):
        shapeDict[dict['id']] = self

class line():
    ...

    def __getstate__(self):
        return {
            "shape1": hex(id(self.shape1)),
            "shape2": hex(id(self.shape2))
        }

    def __setstate__(self, dict):
        self.shape1 = shapeDict[dict['shape1']]
        self.shape2 = shapeDict[dict['shape2']]

At load time I just create a dict of all shapes that I have added with their id as keys and the assign the new reference at load time.