Questions on AWS Elasticsearch Service Cluster Setup

Question

1. How to decide the instance type to use for data nodes and master nodes? Any guidelines for that?
2. Is it necessary to use master nodes in elastic search? How do they help? Can we have a cluster without any master nodes?
3. Does the Elastic search service cluster cost also includes the cost of the master node instances?
4. Can we change the instance types and increase the number of data nodes later without any downtime? Eg. if we found we need more memory or the current instance type are not that useful?

Answer 1

1. It totally depends on your use-case, traffic, types of search queries(real-time or back-office), read-heavy(website search) or write-heavy(log analysis), etc. It's a very open-ended question and it would be the same as you plan the capacity for your other systems. But as master nodes are just used for lightweight cluster-wide actions, it can be much smaller than the data and co-ordinating nodes, where actually heavy lifting of search, aggregation, indexing etc. happens.

2. Yes, its required to use the master node and below is the task performed by master nodes. Although you can mark any node in elasticsearch cluster as the master node and it's not mandatory to have a dedicated master node, but its a good practice to have a dedicated master node for a healthy cluster state.

The master node is responsible for lightweight cluster-wide actions such as creating or deleting an index, tracking which nodes are part of the cluster, and deciding which shards to allocate to which nodes. It is important for cluster health to have a stable master node.

3. It's clearly mentioned on their site. You are charged only for Amazon Elasticsearch Service instance hours, Amazon EBS storage (if you choose this option), and data transfer. Hence if you are creating a dedicated master node then you will have to pay for that as well.

4. You can do both, adding more data nodes to existing ES clusters doesn't require any down-time but if you are changing the instance type, that requires downtime on that node but with the rolling upgrade, you can avoid overall downtime to your ES cluster.

Hope I am able to give a satisfactory answer and let me know if you need more details.