简体繁体 English

如何将日志从 Amazon S3 存储桶导入到 cloudwatch

[英]How to Import Logs From An Amazon S3 Bucket to cloudwatch

原文 2019-06-18 13:33:43 6 2 amazon-web-services/ amazon-s3/ amazon-cloudwatchlogs

I have Exported the aws cloudwatch Log Data to Amazon S3 Using the AWS CLI.我已使用 AWS CLI 将 aws cloudwatch 日志数据导出到 Amazon S3。 Followed kb: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html .遵循知识库： https : //docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html 。

Any idea how we can import the logs saved in s3 for analysing the data?知道我们如何导入保存在 s3 中的日志以分析数据吗？

2 个解决方案

Well, for data analysing I would recommend using Athena,好吧，对于数据分析，我建议使用 Athena，

Regarding your comments a quick search was not able to provide an answer on how to import logs.关于您的评论，快速搜索无法提供有关如何导入日志的答案。

Logs are attached to resources that you spin up, so it sounds kind of impossible to attach logs from a newly generated cluster to the old cluster when it is up, since they are two separate resources with different unique id's etc.日志附加到您启动的资源上，因此在旧集群启动时将日志从新生成的集群附加到旧集群听起来有点不可能，因为它们是具有不同唯一 ID 等的两个独立资源。

Logs are stored in cloudwatch don't have an expiry date even if the resource is deleted.即使资源被删除，存储在 cloudwatch 中的日志也没有到期日期。 So you would still be able to check the logs of the newly generated cluster when it is no longer online (destroyed) export the logs to s3 and analyse them using Athena.因此，当新生成的集群不再在线（销毁）时，您仍然可以检查它的日志，将日志导出到 s3 并使用 Athena 进行分析。

The only way to do this is to use a script of some form to pull the files from S3, and put them into CloudWatch Logs.执行此操作的唯一方法是使用某种形式的脚本从 S3 中提取文件，并将它们放入 CloudWatch Logs。 Lambda would be a good candidate, using an S3 trigger if you wanted to automate it. Lambda 将是一个很好的候选者，如果您想自动化它，请使用 S3 触发器。

But, the better way would be to archive the logs.但是，更好的方法是归档日志。 In a DR scenario as long as your log entries are safe, you're ok.在 DR 场景中，只要您的日志条目是安全的，就可以。 Trying to replay those logs into a new cluster sounds like a bad idea to me.尝试将这些日志重播到新集群中对我来说听起来是个坏主意。