Python serialization issue

Question

I am getting a json message/event with messed up types like:

{
    "AdminCreateUserConfig": {
      "UnusedAccountValidityDays": "7",
      "AllowAdminCreateUserOnly": "true"
    }
    ...
}

I figured out that the best way is to serialize and deserialize using a custom serializer

code:

event = json.loads( json.dumps(event, mySerializer) )

def mySerializer(o):
    if isinstance(o, unicode): 
        if o in ["true", "True"]: return True
        elif o in ["false", "False"]: return False
        else: 
            try:
                return int(o)
            except:
                return o
    else:
        return o.__dict__

But my problem is that after the serialization and deserialization I still get the same strings/unicodes:

AdminCreateUserConfig.UnusedAccountValidityDays: 7, type: <type 'unicode'>
AdminCreateUserConfig.AllowAdminCreateUserOnly: true, type: <type 'unicode'>

What should I change?

TLDR: I am in AWS Lambda, and I have the json Object, this is why I need to do the conversion two more times.

Answer 1

In order to access the key-value pairs of objects as they are being decoded, you need to define an object_pairs_hook method:

def object_pairs_hook(pairs):
    decoded_pairs = []
    for key, val in pairs:
        # python 2/3 compatability support
        try:
            type_ = unicode
        except NameError:
            type_ = str

        if isinstance(val, type_):
            if val.lower() == "true":
                val = True
            elif val.lower() == "false":
                val = False
            else:
                try:
                    val = int(val)
                except ValueError:
                    pass

        decoded_pairs.append((key, val))

    return dict(decoded_pairs)

Which you can then use like this:

>>> import json
>>> json.loads('{ "hi": { "foo": "true", "bar": "2" } }', object_pairs_hook=object_pairs_hook)
{u'hi': {u'foo': True, u'bar': 2}}

The object_hook method (or providing a custom JSONDecoder class) will only be called for objects and will not be called for keys and values of objects.

Using Marshmallow

If your data is well-formed, it would be better to use a library like marshmallow which already handles cases like this. To use it you would define a schema:

import marshmallow

class AdminCreateUserConfig(marshmallow.Schema):
    UnusedAccountValidityDays = marshmallow.fields.Int()
    AllowAdminCreateUserOnly = marshmallow.fields.Bool()

class MySchema(marshmallow.Schema):
    AdminCreateUserConfig = marshmallow.fields.Nested(AdminCreateUserConfig)

And then you can call loads to deserialize:

>>> MySchema().loads('{ "AdminCreateUserConfig": { "UnusedAccountValidityDays": "7", "AllowAdminCreateUserOnly": "true" } }')
{'AdminCreateUserConfig': {'UnusedAccountValidityDays': 7, 'AllowAdminCreateUserOnly': True}}