Read operation right after Elasticsearch index creation causes exception

Question

I try to perform a read opertion on Elasticsearch index right after it was created. Here is simple code to reproduce this situation:

import org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import static java.net.InetAddress.getLoopbackAddress;

public class ElasticIssue {

  static String index = "my_index";

  public static void main(String[] args) {
    final Client c = getClient();
    deleteIndexIfExists(c);
    createIndex(c);
    //refresh(c);
    //flush(c);
    //delay();
    //indexDoc(c);
    getDoc(c);
  }

  static void getDoc(Client client) {
    client.prepareGet(index, "some-type", "1").get();
  }

  static void indexDoc(Client client) {
    client.prepareIndex(index, "another-type", "25").setSource("{}").get();
  }

  static void createIndex(Client client) {
    client.admin().indices().prepareCreate(index).get();
  }

  static void delay() {
    try {Thread.sleep(3000);} catch (InterruptedException e) {}
  }

  static void flush(Client client) {
    client.admin().indices().prepareFlush(index).get();
  }

  private static void refresh(Client client) {
    client.admin().indices().prepareRefresh(index).get();
  }

  static void deleteIndexIfExists(Client client) {
    final IndicesExistsResponse response = client.admin().indices().prepareExists(index).get();
    if (response.isExists()) {
      deleteIndex(client);
    }
  }

  static void deleteIndex(Client client) {
    client.admin().indices().prepareDelete(index).get();
  }

  static Client getClient() {
    final Settings settings = Settings.builder()
        .put("cluster.name", "elasticsearch") //default name
        .put("node.name", "my-node")
        .build();
    return TransportClient.builder()
        .settings(settings)
        .build()
        .addTransportAddress(new InetSocketTransportAddress(getLoopbackAddress(), 9300));
  }
}

And then I get the following error:

Exception in thread "main" NoShardAvailableActionException[No shard available for [get [my_index][some-type][1]: routing [null]]]; nested: RemoteTransportException[[my-node][172.17.0.2:9300][indices:data/read/get[s]]]; nested: IllegalIndexShardStateException[CurrentState[RECOVERING] operations only allowed when shard state is one of [POST_RECOVERY, STARTED, RELOCATED]];
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.perform(TransportSingleShardAction.java:199)
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.onFailure(TransportSingleShardAction.java:186)
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.access$1300(TransportSingleShardAction.java:115)
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction$2.handleException(TransportSingleShardAction.java:240)
    at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:855)
    at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:833)
    at org.elasticsearch.transport.TransportService$4.onFailure(TransportService.java:387)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: RemoteTransportException[[my-node][172.17.0.2:9300][indices:data/read/get[s]]]; nested: IllegalIndexShardStateException[CurrentState[RECOVERING] operations only allowed when shard state is one of [POST_RECOVERY, STARTED, RELOCATED]];
Caused by: [my_index][[my_index][3]] IllegalIndexShardStateException[CurrentState[RECOVERING] operations only allowed when shard state is one of [POST_RECOVERY, STARTED, RELOCATED]]
    at org.elasticsearch.index.shard.IndexShard.readAllowed(IndexShard.java:1035)
    at org.elasticsearch.index.shard.IndexShard.get(IndexShard.java:651)
    at org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:173)
    at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:86)
    at org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:101)
    at org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:44)
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:282)
    at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:275)
    at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33)
    at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
    at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

It seems like Elasticsearch index creation was not complete despite response was already returned. That is a bit frustrating. And if I do any of: delay, index any doc, refresh index, flush index (uncomment any line for this); then read operation performs successfully.

What is the explanation of this behavior? What is a recommended way to make sure that index is ready to work? Listed solutions are found by experiment.

I'am using Elasticsearch 2.3.3 and Java 8. All communication with Elasticsearch is done using Transport protocol (with Java api).

For easier setup here is docker command to get container with all necessary settings:

docker run -p 9200:9200 -p 9300:9300 elasticsearch:2.3.3 -Des.node.name="my-node"

Here is Maven dependency for Elasticsearch Java API:

<dependency>
  <groupId>org.elasticsearch</groupId>
  <artifactId>elasticsearch</artifactId>
  <version>2.3.3</version>
</dependency>

Answer 1

You need to wait till the index is created. This is what you can do to wait till the health of index is in yellow status.

After index creation function call the below function :

static void indexStatusCheck(Client client) {
     ClusterHealthResponse response = client.admin().cluster().prepareHealth().setIndices(index).setWaitForYellowStatus().get();
if (response.getStatus() == ClusterHealthStatus.RED) {
 throw Exception("Index not ready");
}
    }

Then you can proceed with the getDoc() call.