Elasticsearch 5.1 and Docker - How to get networking configured properly to reach Elasticsearch from the host

Question

Using Elasticsearch:latest (v5.1) from the Docker public repo, I created my own image containing Cerebro. I am now attempting to get Elasticsearch networking properly configured so that I can connect to Elasticsearch from Cerebro. Cerebro running inside of the container I created, renders properly on my host at: http://localhost:9000 .

After committing my image, I created my Docker container with the following:

sudo docker run -d -it --privileged --name es5.1 --restart=always \
-p 9200:9200 \
-p 9300:9300 \
-p 9000:9000 \
-v ~/elasticsearch/5.1/config:/usr/share/elasticsearch/config \
-v ~/elasticsearch/5.1/data:/usr/share/elasticsearch/data \
-v ~/elasticsearch/5.1/cerebro/conf:/root/cerebro-0.4.2/conf \
elasticsearch_cerebro:5.1 \
/root/cerebro-0.4.2/bin/cerebro

my elasticsearch.yml in ~/elasticsearch/5.1/config currently has the following network and discovery entries specified:

network.publish_host: 192.168.1.26
discovery.zen.ping.unicast.hosts: ["192.168.1.26:9300"]

I have also tried 0.0.0.0 and not specifying the values to default to the loopback for these settings. In addition, I've tried specifying network.host with a combination of values. No matter how I set this, elasticsearch logs on startup:

[info] play.api.Play - Application started (Prod)
[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
[error] p.c.s.n.PlayDefaultUpstreamHandler - Cannot invoke the action
java.net.ConnectException: Connection refused: localhost/127.0.0.1:9200
… cascading errors because of this connection refusal...

No matter how I set the elasticsearch.yml networking, the error message on Elasticsearch startup does not change. I verified that the elasticsearch.yml is being picked-up inside of the Docker container. Please let me know were I'm going wrong with this configuration.

Answer 1

Well, it looks like I"m answering my own question after a days-worth of battle with this! The issue was that elasticsearch wasn't started inside of the container. To determine this, I got a terminal into the container:

docker exec -it es5.1 bash

Once in the container, I checked service status:

service elasticsearch status

To this, the OS responded with:

[FAIL] elasticsearch is not running ... failed!

I started it with:

service elasticsearch start

I add a single script that I'll call from docker run to start elasticsearch and cerebro and that should do the trick. However, I would still like to hear if there is a better way to configure this.

Answer 2

I made a github docker-compose repo that will spin up a elasticsearch, kibana, logstash, cerebro cluster

https://github.com/Shuliyey/elkc

========================================================================

On the other hand, in regard to the actual problem (elasticsearch_cerebro not working).

To get the elasticsearch and cerebro working in one docker container. Need to use supervisor

https://docs.docker.com/engine/admin/using_supervisord/

will update with more details

Answer 3

No need to use supervisor at all. A very simple way to solve this is to use docker-compose and bundle Elasticsearch and Cerebro together, like this:

docker-compose.yml:

version: '2'

services:

  elasticsearch:
    build: elasticsearch
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/data:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
      - "9300:9300"
    environment:
      ES_JAVA_OPTS: "-Xmx1500m -Xms1500m"
    networks:
      - elk

  cerebro:
    build: cerebro
    volumes:
      - ./cerebro/config/application.conf:/opt/cerebro/conf/application.conf
    ports:
      - "9000:9000"
    networks:
      - elk
    depends_on:
      - elasticsearch

networks:
  elk:
    driver: bridge

elasticsearch/Dockerfile:

FROM docker.elastic.co/elasticsearch/elasticsearch:5.5.1

cerebro/Dockerfile:

FROM yannart/cerebro

Then you run docker-compose build and docker-compose up . When everything is started, you can access ES at http://localhost:9200 and Cerebro at http://localhost:9000