docker-elk - how is it persisting elasticsearch index?

Question

I'm just getting to grips with Docker and docker-compose, trying to create a development environment for Elasticsearch which I will deploy later.

I've been using docker-elk as a reference, and I've managed to create a working Elasticsearch container, seed it, and use it in my project.

As I understand it, Docker containers don't persist data, unless you use the Volumes API and create a volume outside the container that the container then accesses (read that here ).

However docker-elk only uses Volumes to share a config yml file, but somehow my elastic indices are persisting when I bring the container down and up again.

From the docker-elk readme:

The data stored in Elasticsearch will be persisted after container reboot but not after container removal.

Can someone please explain what part of the below configuration is allowing the docker container to persist the index?

docker-compose.yml

version: '2'

services:
  elasticsearch:
    build:
      context: build/elasticsearch/
    volumes:
      - ./build/elasticsearch/config.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
    ports:
      - "9200:9200"
      - "9300:9300"
    environment:
      ES_JAVA_OPTS: "-Xmx256m -Xms256m"
    networks:
      - elk

networks:
  elk:
    driver: bridge

build/elasticsearch/Dockerfile

FROM docker.elastic.co/elasticsearch/elasticsearch-oss:6.0.0

build/elasticsearch/config.yml

cluster.name: "docker-cluster"
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1
discovery.type: single-node

Answer 1

As you may know, a container is a sandbox. It has a filesystem with a structure very identical to a typical linux OS. The container only sees those files and folders that are in this filesystem.

The process running inside the container writes it data and config to files in this filesystem. This process is unaware that it is running in a container or on a VM. Thus the data is persisted in files and folder in this filesystem.

Now when you remove a container using docker rm ... those files are deleted with the container and thus you lose the data unless you use volumes which backup this data on the host.

On the other hand, stopping and starting the container does not remove the container files and thus the data is still there when you restart the container.

Answer 2

To supplement the accepted answer, for anyone who is looking for how to persist the data. Add a volume as mentioned in the question.

version: '3'

services:
  
  elasticsearch: # Elasticsearch Instance
    container_name: es-search
    image: docker.elastic.co/elasticsearch/elasticsearch:6.1.1
    volumes: # Persist ES data in seperate "esdata" volume
      - esdata:/usr/share/elasticsearch/data
    environment:
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.type=single-node
    ports: # Expose Elasticsearch ports
      - "9300:9300"
      - "9200:9200"

volumes: # Define seperate volume for Elasticsearch data
  esdata: ./my/esdata # path of your persisted data here

I found a guide for elastic docker here: https://blog.patricktriest.com/text-search-docker-elasticsearch/

Answer 3

可以使用以下命令观察UUID 中的索引及其映射。

curl 'localhost:9200/_cat/indices?v'