Dynamo Db流直接进入Elastic Search，而无需其他中间层

Question

Can we directly stream dynamo db data to AWS elastic search service without using logstash because using logstash will incur extra cost? In all the articles that I have read online it was either with logstash or with lambda we can achieve this.

Answer 1

It seems that you can: https://aws.amazon.com/blogs/compute/indexing-amazon-dynamodb-content-with-amazon-elasticsearch-service-using-aws-lambda/

I've used DynamoDB on AWS in the past, setup up streams to push changes from DynamoDB to an endpoint and then used Logstash to read from the endpoint and write the changes to ES. It seems that now you can use a Lambda on the streams to write to ES without needing Logstash.

That said, the Logstash approach would also migrate all the existing data in the dynamoDB table on start. The streams/lambda approach listed above apparently doesn't do this. To provide that functionality the article mentions setting up an additional Kinesis Stream, using that as a second input to your ES writer lambda, then running some python code or similar to load all the existing data from the DB into the kinesis stream.

This seems like a lot of complexity and potentially more costly than just using Logstash to cover both scenarios.