如何配置Elasticsearch从Amazon SQS提取消息

Question

In my current scenario , i am having ElasticSearch on a ubuntu 14.04 machine and a amazon SQS queue holding some ".json" messages in the queue. I want to pull the messages from amazon SQS via ES so that it can use the .json messages to index it.

What should i be looking for, how exactly should i be proceeding? I am looking for something in lines of python's boto code.

NOTE: I am not interested in using logstash for this

Answer 1

Amazon SQS provides an API to receive messages: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/ReceiveMessage.html

ElasticSearch provides an API to index messages: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index-doc.html

I am not sure if ES can automatically receive messages from SQS, but one feasible solution is a simple broker between SQS and ES to receive and index.