How to restrict certain indices to a set of machines in an Elasticsearch cluster

Question

We have 2x64GB machines and 2x16GB machines in a cluster and we are creating one index per day. The requirement is to move the new index to the 64 GB machines and the old ones to the 16GB machines. How can this be done with elasticsearch?

Answer 1

You need to "tag" your nodes and specify which is which by using a node.tag property: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#shard-allocation-filtering .

And you specify node.tag: large1 and node.tag: large2 for each 64GB nodes (one for each).

Then, when you create your index you specify how you want it to be allocated by using "index.routing.allocation.include.tag" : "large1,large2" (these are the 64GB nodes). When you decide to move the index from two nodes to the other two nodes, then you update the property above and specify "small1,small2" for example (smaller 16GB nodes).

I would suggest having a look at Curator , as well, that can help you change these settings.

Answer 2

This is a 2 step process

1. Tag your node with key value pairs. For eg: , for lesser machines tag them as boxType:small and for better machine tag them as boxType:big.
2. Next apply the index filter rule using the settings API as below -

  curl -XPUT localhost:9200/index-name/_settings -d '{ "index.routing.allocation.include.tag" : "big" }'