将日志从 AWS Cloudwatch 日志组发送到 Opendistro EFK

Question

Is there a way to ship logs from AWS Cloudwatch log group to Opendistro EFK? I am using opendistro which has the elasticsearch version 7.10.2, that has been provisioned using helm in an EKS cluster. I tried with functionbeat 7.10.2, but once I tried shipping a sample cloudwatch log data, it throws the error:

2021-10-21T19:17:32.605Z    ERROR   [publisher_pipeline_output] pipeline/output.go:154  Failed to connect to backoff(elasticsearch(https://******.io:443)): Connection marked as failed because the onConnect callback failed: 169.254.40.221 requires the default distribution of Elasticsearch. Please update to the default distribution of Elasticsearch for full access to all free features, or switch to the OSS distribution of 169.254.40.221.
2021-10-21T19:17:32.605Z ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://******.io:443)): Connection marked as failed because the onConnect callback failed: 169.254.40.221 requires the default distribution of Elasticsearch. Please update to the default distribution of Elasticsearch for full access to all free features, or switch to the OSS distribution of 169.254.40.221.

Is there any compatible version of functionbeat with opendistro Elasticsearch 7.10.2? Is there any other beat that can be used to ship logs from AWS cloudwatch to Open distro EFK?

Answer 1

using kinesis and self hosted logstash can work,there is cost using kinesis