Using cattrs / attrs where attr name does not match keys to create an object

Question

I am looking at moving to cattrs / attrs from a completely manual process of typing out all my classes but need some help understanding how to achieve the following.

This is a single example but the data returned will be varied and sometimes not with all the fields populated.

data = {
  "data": [
    {
      "broadcaster_id": "123",
      "broadcaster_login": "Sam",
      "language": "en",
      "subscriber_id": "1234",
      "subscriber_login": "Dave",
      "moderator_id": "12345",
      "moderator_login": "Tom",
      "delay": "0",
      "title": "Weekend Events"
    }
  ]
}

@attrs.define
class PartialUser:
    id: int
    login: str

@attrs.define
class Info:
    language: str
    title: str
    delay: int
    broadcaster: PartialUser
    subscriber: PartialUser
    moderator: PartialUser

So I understand how you would construct this and it works perfectly fine with 1:1 mappings, as expected, but how would you create the PartialUser objects dynamically since the names are not identical to the JSON response from the API?

instance = cattrs.structure(data["data"][0], Info)

Is there some trick to using a converter? This would need to be done for around 70 classes which is why I thought maybe cattrs could modernise and simplify what I'm trying to do.

thanks

Answer 1

Here's one possible solution.

This is the strategy: we will customize the structuring hook by wrapping it. The default hook expects the keys in the input dictionary to match the structure of the class, but here this is not the case. So we'll substitute our own structuring hook that does a little preprocessing and then calls into the default hook.

The default hook for an attrs class cls can be retrieved like this:

from cattrs import Converter
from cattrs.gen import make_dict_structure_fn

c = Converter()
handler = make_dict_structure_fn(cls, c)

Knowing this, we can implement a helper function thusly:

def group_by_prefix(cls: type, c: Converter, *prefixes: str) -> None:
    handler = make_dict_structure_fn(cls, c)

    def prefix_grouping_hook(val: dict[str, Any], _) -> Any:
        by_prefix = {}
        for key in val:
            if "_" in key and (prefix := (parts := key.split("_", 1))[0]) in prefixes:
                by_prefix.setdefault(prefix, {})[parts[1]] = val[key]
        return handler(val | by_prefix, _)

    c.register_structure_hook(cls, prefix_grouping_hook)

This function takes an attrs class cls , a converter, and a list of prefixes. Then it creates a hook and registers it with the converter for the class cls . Inside, it does a little bit of preprocessing to beat the data into the shape cattrs expects.

Here's how you'd use it for the Info class:

>>> c = Converter()
>>> group_by_prefix(Info, c, "broadcaster", "subscriber", "moderator")
>>> print(c.structure(data["data"][0], Info))
Info(language='en', title='Weekend Events', delay=0, broadcaster=PartialUser(id=123, login='Sam'), subscriber=PartialUser(id=1234, login='Dave'), moderator=PartialUser(id=12345, login='Tom'))

You can use this approach to make the solution more elaborate as needed.