将 DynamoDb 指标日志导出到 S3 或 CloudWatch

Question

I'm trying to use DynamoDB metrics logs in an external observability tool.

To do that, I'll need to get these log data from S3 or CloudWatch log groups (not from Insights or CloudTrail).

For this reason, if there isn't a way to use CloudWatch, I'll need to export these metric logs from DynamoDb to S3, and from there export to CloudWatch or try to get those data directly from S3.

Do you know this is possible?

Answer 1

You could try using Logstash, it has a plugin for Cloudwatch and S3:

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-cloudwatch.html

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html

Answer 2

AWS puts DynamoDB metrics (table operation, table, and account) over CloudWatch metrics. Also, you can create as many metrics as you need. If you use Python, you can read it with boto3. The CloudWatch client has this method: get_metric_data

Try this with your metrics:

cloudwatch_client = boto3.client('cloudwatch')
yesterday = date.today() - timedelta(days=1)
today = date.today()
response = cloudwatch_client.get_metric_data(
    MetricDataQueries=[
        {
            'Id': 'some_request',
            'MetricStat': {
                'Metric': {
                    'Namespace': 'DynamoDB',
                    'MetricName': 'metric_name',
                    'Dimensions': []
                },
                'Period': 3600,
                'Stat': 'Sum',
            }
        },
    ],
    StartTime=datetime(yesterday.year, yesterday.month, yesterday.day),
    EndTime=datetime(today.year, today.month, today.day),
)
print(response)