Parsing AWS Lambda Python Function Response from Boto3 Cost Explorer Client

Question

I am writing a Lambda Function in Python 3.6 to query out specific conditions from the Cost Explorer API, it will eventually be invoked by API Gateway so I want to be able to send back a pared down response, as well as take that same response and persist it into S3.

I have the overall functionality working correctly, I was hoping to shortcut parsing and just crawl the response, but that was not working with Glue and Athena. The basic functioning code is below:

import boto3
import json

def handler(event,context):
    client = boto3.client('ce')
    s3 = boto3.resource('s3')
    object = s3.Object('s3-bucket', 'path/object.json')
    response = client.get_cost_and_usage(
    TimePeriod={
        'Start': '2019-01-01',
        'End': '2019-07-01'
    },
    Granularity='MONTHLY',
    Metrics=[
        'UnblendedCost',
    ],
    GroupBy=[
        {
            'Type': 'DIMENSION',
            'Key': 'SERVICE'
        },
    ],
    )
    object.put(Body=json.dumps(response).encode())
    return str (response)

It gives a response like this according to the docs

{
    'NextPageToken': 'string',
    'GroupDefinitions': [
        {
            'Type': 'DIMENSION'|'TAG',
            'Key': 'string'
        },
    ],
    'ResultsByTime': [
        {
            'TimePeriod': {
                'Start': 'string',
                'End': 'string'
            },
            'Total': {
                'string': {
                    'Amount': 'string',
                    'Unit': 'string'
                }
            },
            'Groups': [
                {
                    'Keys': [
                        'string',
                    ],
                    'Metrics': {
                        'string': {
                            'Amount': 'string',
                            'Unit': 'string'
                        }
                    }
                },
            ],
            'Estimated': True|False
        },
    ]
}

Which looks something like this when I run my function (VS Code did this spacing) :

{
    "GroupDefinitions": [
        {
            "Type": "DIMENSION",
            "Key": "SERVICE"
        }
    ],
    "ResultsByTime": [
        {
            "TimePeriod": {
                "Start": "2019-01-01",
                "End": "2019-02-01"
            },
            "Total": {
                "UnblendedCost": {
                    "Amount": "0",
                    "Unit": "USD"
                }
            },
            "Groups": [],
            "Estimated": true
        },
        {
            "TimePeriod": {
                "Start": "2019-02-01",
                "End": "2019-03-01"
            },
            "Total": {
                "UnblendedCost": {
                    "Amount": "0",
                    "Unit": "USD"
                }
            },
            "Groups": [],
            "Estimated": true
        },
        {
            "TimePeriod": {
                "Start": "2019-03-01",
                "End": "2019-04-01"
            },
            "Total": {
                "UnblendedCost": {
                    "Amount": "0",
                    "Unit": "USD"
                }
            },
            "Groups": [],
            "Estimated": false
        },
        {
            "TimePeriod": {
                "Start": "2019-04-01",
                "End": "2019-05-01"
            },
            "Total": {},
            "Groups": [
                {
                    "Keys": [
                        "AWS CloudTrail"
                    ],
                    "Metrics": {
                        "UnblendedCost": {
                            "Amount": "0.032953",
                            "Unit": "USD"
                        }
                    }
                },
                {
                    "Keys": [
                        "AWS CodeCommit"
                    ],
                    "Metrics": {
                        "UnblendedCost": {
                            "Amount": "0",
                            "Unit": "USD"
                        }
                    }
                },
                {
                    "Keys": [
                        "AWS Config"
                    ],
                    "Metrics": {
                        "UnblendedCost": {
                            "Amount": "10.148",
                            "Unit": "USD"
                        }
                    }
                },
                {
                    "Keys": [
                        "AWS Elemental MediaStore"
                    ],
                    "Metrics": {
                        "UnblendedCost": {
                            "Amount": "0",
                            "Unit": "USD"
                        }
                    }
                }
            ],
            "ResponseMetadata": {
                "RequestId": "1d149b43-3b7b-46cb-973a-6b0e9adfbe14",
                "HTTPStatusCode": 200,
                "HTTPHeaders": {
                    "x-amzn-requestid": "1d149b43-3b7b-46cb-973a-6b0e9adfbe14",
                    "content-type": "application/x-amz-json-1.1",
                    "content-length": "9310",
                    "date": "Sun, 07 Jul 2019 00:00:00 GMT"
                },
                "RetryAttempts": 0
            }

I am trying to parse out only the information I get from Groups, both as the printed response when I proxy it back to API Gateway and also when I persist it to S3 (or Dynamo) to save reports and eventually add some analytics layering to this. I modified the end of my function code to this:

...Lambda Code...
object.put(Body=json.dumps(response['ResultsByTime'][0]['Groups']['Keys']).encode())
    return str (response['ResultsByTime'][0]['Groups']['Keys'])

That did not work, and now I am getting this error in CloudWatch Logs

list indices must be integers or slices, not str: TypeError
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 25, in handler
print(response['ResultsByTime'][0]['Groups']['Keys'])
TypeError: list indices must be integers or slices, not str

Is there something obvious I am doing wrong? Am I not allowed to parse out only a specific array in the body? Thanks for your help in advance!

Answer 1

The first entry in ResultsByTime has an empty Group . Therefore, it is not possible to get Keys from it.

Instead, use:

response['ResultsByTime'][0]['Groups'].get('Keys', '')

This will return an empty string if Keys doesn't exist.