Elasticsearch in production with kubernetes

Question

I am working on product in which we are using elasticsearch for search. Our production setup is in K8S (1.7.7) and we are able to scale it pretty well. Only thing I am not sure about is whether we should be hosting elasticsearch in k8s (it can go on dedicated host as well using label selector nodes) or it is advisable to host elasticsearch on VM than docker. Our data set size is 2-3 GB and would go further. But this is the benchmark we can consider. And elasticsearch cluster I am planning to have ti is - 3 master (with 2 as eligible master), one client node, and one data node. We can scale datanode and client node as data increases. Is anyone did this before? thanks in advance.

Answer 1

IMO the best resource for Elasticsearch on Kubernetes is https://github.com/pires/kubernetes-elasticsearch-cluster

Note that while there are official Docker containers, no official solution for orchestration is being provided at the moment. This is currently covered by the community only.

3 master (with 2 as eligible master)

This doesn't make much sense. You'll want 3 master eligible nodes with the setting discovery.zen.minimum_master_nodes: 2 and one of the 3 nodes will be the actual master.