Rabbitmq plugin for ElasticSearch 2.x

Question

I have installed elastic search 2.3.4 in ubuntu 14.04

http://10.xx11:9200/

{
  "name" : "Everett Thomas",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.3.4",
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
    "build_timestamp" : "2016-06-30T11:24:31Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

i have rabbitmq server running on different system (10.xx12). Now, i am trying to connect rabbitmq server with elasticsearch so that all the messages published in the queue will be consumed and inserted into elastic search.

/etc/logstash/conf.d/rabbit.conf contains

rabbitmq {
    host => "10.x.x.12"
    queue => "integration"
    durable => true
    key => "#"
    exchange => "integ"
    threads => 3
    prefetch_count => 50
    port => 5672
    user => "guest"
    password => "guest"
}

Due to plugin issue logstash is not coming up.

service logstash configtest
Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after  {:level=>:error}

I believe we need to install plugin for elastic search . In this website i dont see the plugin available for elastic search > 2.0.

Can someone help in finding the apt plugin so that messages will be published in the queue.

Answer 1

In your rabbit.conf file, you need to add the rabbitmq input to the input {} section. You also need an output{} section with the elasticsearch output to send the docs to your ES server.

input {
  rabbitmq {
    host => "10.x.x.12"
    queue => "integration"
    durable => true
    key => "#"
    exchange => "integ"
    threads => 3
    prefetch_count => 50
    port => 5672
    user => "guest"
    password => "guest"
  }
}
output {
   elasticsearch {
       hosts => ["10.x.x.11:9200"]
       index => "your_index"
       document_type => "your_type"
   }
}