如何使用类型信息和枚举将 JSON 转换为 object？

Question

What's the best way to marshal some JSON into a strongly-typed Python object that offers more guarantees about the data than just converting into a dict -alike?

I have some JSON from an external API that looks like this:

{
    "Name of Event": {
        "start": "2021-01-01 00:00:00",
        "event_type": 1
    },
    "Another Event": {
        "start": "2021-01-01 00:00:00",
        "event_type": 2,
    }
}

(It's important to know that the actual JSON I'm working with is a lot more complicated (and deeply nested) but basically it's a very well-structured thing with known types all the way down.)

I can do something as simple as:

for name, event in json.loads(data):
     do_things(name, event['start'], event['event_type'])

but this feels fairly woolly and doesn't give my program much to go on in terms of typechecking either at write time or run time.

In the code where I handle this JSON I'd like to work with something with the right types. But I don't want to write a ton of boilerplate.

I can do something very very explicit like:

DATE_FORMAT = "%y-%m-%d %H:%M:%S"

class EventType:
    FREE_FOR_ALL = 1
    CLOSED_REGISTRATION = 2

class Event:
    __slots__ = ["start", "event_type"]

    start: datetime.datetime
    event_type: EventType

    def __init__(self, start, event_type):
        self.start = datetime.datetime.strptime(start, DATE_FORMAT)
        self.event_type = EventType(event_type)

    def __str__(self):
        return str(self.__dict__)

    def __repr__(self):
        return str(self.__dict__)

APIResponse = Dict[str, Event]

for name, raw_event in data:
    event = Event(**data)
    do_things(name, event)

This is fine as far as it goes but once you have a dozen classes with a dozen attributes each it starts to look like a lot of boilerplate. In particular I feel like I'm defining each property twice, violating DRY. Once on the class and once in __init__ .

(I'm also slightly concerned about this being a bit "brittle" for situations like the API adding a new option in any given enum, etc. but that's a much lesser concern as I would expect API changes to require changes to my code.)

I am wondering if there is any magic I can use that will make it so I only have to define each field in one place but still get good typechecking and runtime guarantees that the data is in the form I expect?

I took a look at dataclasses but it seems like I am not able to interfere with the simple string/int inputs of the JSON into enums, datetimes, etc. I could use InitVar to mark a lot of the inputs as " __init__ only" and then use __post_init__ to populate other fields with different names with the strongly-typed version of my data. But the "right" name for most of these fields is the name it already has in the JSON (I don't want to have to write event.event_type_typed_version ).

Ideally I'd write something like this:

@magic_annotation
class Event:
    start: datetime.datetime
    event_type: EventType

for name, raw_event in data:
    event = Event(**data)
    do_things(name, event)

and not need anything else. Does magic_annotation exist? Is there a completely different way to go about this?

Answer 1

Sounds like you're after pydantic .

from datetime import datetime
from enum import Enum
from pydantic import BaseModel

class EventType(Enum):
    FREE_FOR_ALL = 1
    CLOSED_REGISTRATION = 2


class Event(BaseModel):
    start: datetime
    event_type: EventType

event = Event.parse_obj({
    "start": "2021-01-01 00:00:00",
    "event_type": 1
})

print(repr(event))
# Event(start=datetime.datetime(2021, 1, 1, 0, 0), event_type=<EventType.FREE_FOR_ALL: 1>)

Pydantic automatically converts your inputs based on the type annotations of your classes' attributes. For datetimes, it supports the standard ISO 8601 formats. For enums, it converts automatically from the enum values. Check it out! (Not affiliated, just a fan.)