使用boto3在dynamodb put_item中的Unicode解码错误

Question

I'm trying to store API passwords and the like in a dynamodb table, encrypted with AES, and I'm having trouble storing it. I have a dict with data that I'm writing.

for key in endpoint:
        if 'password' in key or 'security' in key or 'secret' in key:
            #encrypt the value
            obj = AES.new(
                landlord_registry['default_crypt_salt'],
                AES.MODE_CFB,
                landlord_registry['default_crypt_iv'])
            endpoint[key] = {'B' : obj.encrypt(endpoint[key])}
        else:
            endpoint[key] = {'S' : endpoint[key]}

The resulting dict looks like this:

{'mvp_version': {'S': '0.3'}, 'endpoint_details': [{u'time_interval': {'S': u'30'}, u'username': {'S': u'blahblah@blahblah.org'}, u'primary_key': {'S': u'Id'}, u'security_token': {'B': '&\\xd9\\t\\x7b\\x9...'}, u'service_name': {'S': u'blahblah'}, u'enabled': {'S': u'1'}, u'endpoint_id': {'S': u'SFORG0'}, u'password': {'B': 'K9\\x958,\\x31.... '}}, {u'username': {'S': u'usernamelala'}, u'primary_key': {'S': u'ID__'}, u'database': {'S': u'z8_dev1'}, u'service_name': {'S': u'sdasda'}, u'enabled': {'S': u'1'}, u'host_name': {'S': u'lalala.rds.amazonaws.com'}, u'endpoint_id': {'S': u'MYSQL1'}, u'password': {'B': 'E##\\xe2n....'}}]}

The error is: Dynamo Insert Error: (, UnicodeDecodeError('utf8', ... , 1, 2, 'invalid continuation byte')

But from what I can tell, since I've used {'B' : 'encoded text'} it should try to store this as bytes not unicode text.

Answer 1

I needed to use the Binary(text) wrapper in Python 2 in order to store the binary text in dynamodb. So the line:

endpoint[key] = {'B' : obj.encrypt(endpoint[key])}

should look like:

endpoint[key] = {'B' : Binary(obj.encrypt(endpoint[key]))}