Does Elasticsearch check for the number of nodes available in the cluster before creating the shards(in cases when default shard size is selected)

Question

i have a cluster with single node in it.. i had created an index A with default shard size(ie in elasticsearch.yml file the value of index.number_of_shards: 1 ). When i listed all my shards, i could see single shard for index A . After this i changed the value of index.number_of_shards: 4 in elasticsearch.yml and then created another index B . again when i listed all my shards in the cluster i could only see single shard created for index B instead of 4 shards. Does Elasticsearch check for the total number of nodes present in a cluster before creating the index and assigning the shards(in my case i had not specified any no of shards while creating the index B , so i was expecting total of 4 shards to be created for my index). can you help me with this?

Answer 1

The index configuration on elasticsearch.yml is being deprecated in favor of passing this configuration on the index settings/mapping instead.

So what you have to do is to remove index configurations from the elasticsearch.yml file and pass them using index settings or template. From elastic docs:

curl -XPUT 'http://localhost:9200/twitter/' -d '{
"settings" : {
    "index" : {
        "number_of_shards" : 3, 
        "number_of_replicas" : 2 
    }
}

Using this approach you can create different configurations for each index.

Note: elasticsearch.yml is a "global" / static configuration file of elasticsearch that is read during startup and if you change it it will not affect current running instances.

Answer 2

Does Elasticsearch check for the total number of nodes present in a cluster before creating the index and assigning the shards

No. It will happily create many shards for the same index on the same node. Most likely you didn't restart ES after changing elasticsearch.yml. It's not read live, so any changes to it require a restart to take effect.