The data is getting lost whenever I restart the docker/elk image

Question

I'm using docker/elk image to display my data in kibana dashboard (Version 6.6.0) and It works pretty good. I started the service like using below command.

Docker Image git repo :

 https://github.com/caas/docker-elk

Command:

    sudo docker-compose up --detach

Expecting that it will run background, and did as expected. After two days the server up and running the and third day the kibana alone getting stopped. and Used below command to make it up and running.

    sudo docker run -d <Docer_image_name>

It's up and running when I use docker ps command. But when I tried to hit the kibana server in chrome browser it says not reachable.

So I just used to below command to restart the service.

    sudo docker-compose down

After that I can see kibana server in chrome browser which is up and running but I do see all my data is lost.

I used below URL in jenkins to collect the data.

`http://hostname:9200/ecdpipe_builds/extern`al 

Any idea how can I resolve this issue?

Answer 1

I did not see the persistent storage configuration the image you mentioned in their GitHub docker-compose file.

This is common to lost data in case of docker container if you did not provide persistent storage configuration. so docker-compose down may cause to lost you data if there is no persistent configuration docker-compose file.

Persisting log data

In order to keep log data across container restarts, this image mounts /var/lib/elasticsearch — which is the directory that Elasticsearch stores its data in — as a volume.

You may however want to use a dedicated data volume to persist this log data, for instance to facilitate back-up and restore operations.

One way to do this is to mount a Docker named volume using docker's -v option, as in:

$ sudo docker run -p 5601:5601 -p 9200:9200  -p 5044:5044 \
    -v elk-data:/var/lib/elasticsearch --name elk sebp/elk

This command mounts the named volume elk-data to /var/lib/elasticsearch (and automatically creates the volume if it doesn't exist; you could also pre-create it manually using docker volume create elk-data).

So you can set these paths in your docker-compose file accordingly. Here is the link that you can check elk-docker-persisting-log-data

Answer 2

Use docker volume or file location as persistant space