Creating Pydantic Model for large nested Parent, Children complex JSON file

Question

I have a JSON file that has a number of objects that have parent, child relationships. This is what the JSON looks like at a high-level.

{
    "objectType1": {
        "objectType1ID1": {
            "parent": {},
            "children": [{"id":"objectType2ID2"}],
            "properties": {}
        },
        "objectType1ID2": {
            "parent": {},
            "children": [{}],
            "properties": {}
        }
    },
    "objectType2": {
        "objectType2ID1": {
            "parent": {},
            "children": [],
            "properties": {}
        },
        "objectType2ID2": {
            "parent": {"id":"objectType1ID1"},
            "children": [],
            "properties": {}
        }
    }
}

The first level keys contain the various types of objects and then within each type there are any number of objects of that type that contain parent, children (list), and properties keys. Each object can have at most one parent object or many children objects in a list.

I am trying to build a Pydantic model for this structure. Below is what I have so far but I am new to Pydantic and I am stuck.

from pydantic import BaseModel
from typing import List, Optional

class ObjectParent(BaseModel):
    id: Optional[str]

class ObjectChild(BaseModel):
    id: Optional[str]

class Properties(BaseModel):
    name: Optional[str]
    description: Optional[str]


class Object(BaseModel):
    id: str
    parent: Optional['ObjectParent'] = None
    children: List['ObjectChild'] = []
    properties: Properties

Answer 1

I took at a stab at solving this, some advanced usage of pydantic is required.

TLDR

Let's cut to the chase -> here's the functionality you want to utilize:

from typing import Mapping
from pydantic import BaseModel

class JsonObject(BaseModel):
    __root__: Mapping[str, Mapping[str, int]]

# Allows instantiating by parsing an object:
my_obj = JsonObject.parse_obj({
    "a": {"one": 1, "two": 2},
    "b": {"three": 3},
    "c": {}
})

>> my_obj.__root__
{'a': {'one': 1, 'two': 2}, 'b': {'three': 3}, 'c': {}}

Now let's dive into it

Let's try to model a single "root" object, at the depth of your nested structure:

"objectType1ID1": {
  "parent": {},
  "children": [
   {
    "id": "objectType2ID2"
   }
  ],
  "properties": {}
 }

The following pydantic models should work:

class AssociatedObject(BaseModel):
    """
    Represents an associated object, whether
    that is a parent or a child.
    """
    id: Optional[str]


class ObjectProperties(BaseModel):
    name: Optional[str]
    description: Optional[str]


class BaseObject(BaseModel):
    """
    Represents an object at the furthest depth
    of the nested structure
    """
    parent: AssociatedObject = AssociatedObject()
    children: List[AssociatedObject] = [AssociatedObject()]
    properties: ObjectProperties = ObjectProperties()

If we instantiate with empty values, we should get what is represented in your json object:

>> BaseObject().dict(exclude_none=True)
{'parent': {}, 'children': [{}], 'properties': {}}

Now for the hard part – how to define the parent keys?

Create a class which can store our object and its type name ( objectType1ID1 ) as a dynamic dictionary:

from typing import Mapping

class JsonObject(BaseModel):
    __root__: Mapping[str, Mapping[str, BaseObject]]

If we instantiate, we see it works as expected:

json_object = JsonObject(__root__=
    {
        "objectType1": {
            "objectType1ID1": BaseObject(parent=AssociatedObject(id="objectType2ID2")),
            "objectType1ID2": BaseObject()
        },
        "objectType2": {
            "objectType2ID1": BaseObject(),
            "objectType2ID2": BaseObject(parent=AssociatedObject(id="objectType1ID1"))
        }
    }
)

parent_object.dict(exclude_none=True)['__root__']

{
 "objectType1": {
  "objectType1ID1": {
   "parent": {
    "id": "objectType2ID2"
   },
   "children": [
    {}
   ],
   "properties": {}
  },
  "objectType1ID2": {
   "parent": {},
   "children": [
    {}
   ],
   "properties": {}
  }
 },
 "objectType2": {
  "objectType2ID1": {
   "parent": {},
   "children": [
    {}
   ],
   "properties": {}
  },
  "objectType2ID2": {
   "parent": {
    "id": "objectType1ID1"
   },
   "children": [
    {}
   ],
   "properties": {}
  }
 }
}

Finally: You can now parse your initial nested object successfully:

# Object provided in original Q
my_initial_object = {
    "objectType1": {
        "objectType1ID1": {
            "parent": {},
            "children": [{"id":"objectType2ID2"}],
            "properties": {}
        },
        "objectType1ID2": {
            "parent": {},
            "children": [{}],
            "properties": {}
        }
    },
    "objectType2": {
        "objectType2ID1": {
            "parent": {},
            "children": [],
            "properties": {}
        },
        "objectType2ID2": {
            "parent": {"id":"objectType1ID1"},
            "children": [],
            "properties": {}
        }
    }
}

# This works
my_json_object = JsonObject.parse_obj(my_initial_object)