How to connect to AWS Elasticsearch?

Question

I'm trying to connect to AWS Elasticsearch but I always get the following error:

Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:278)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:197)
at org.elasticsearch.client.transport.support.InternalTransportClient.execute(InternalTransportClient.java:106)
at org.elasticsearch.client.support.AbstractClient.index(AbstractClient.java:98)
at org.elasticsearch.client.transport.TransportClient.index(TransportClient.java:334)
at org.elasticsearch.action.index.IndexRequestBuilder.doExecute(IndexRequestBuilder.java:313)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at com.c_backendcrawler.utility.ElasticSearch.uploadObject(ElasticSearch.java:25)
at com.c_backendcrawler.Start.main(Start.java:34)

My code is following:

 //Create Client
    Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "zencubes-search").put("node.name","Darkhawk").build();
    TransportClient transportClient = new TransportClient(settings);
    transportClient.addTransportAddress(new InetSocketTransportAddress(
            "x.x.x.x",9300));
    return transportClient;

Output from AWS Elasticsearch:

    {
status: 200,
name: "Darkhawk",
cluster_name: "817880037706:zencubes-search",
version: {
number: "1.5.2",
build_hash: "62ff9868b4c8a0c45860bebb259e21980778ab1c",
build_timestamp: "2015-04-27T09:21:06Z",
build_snapshot: false,
lucene_version: "4.10.4"
},
tagline: "You Know, for Search"
}

I tried to curl ( https://search-zencubes-search-xxxxxxxx.eu-west-1.es.amazonaws.com/ ) and it works - but not on port 9300. What am I doing wrong here?

Answer 1

The native transport protocol is not support using AWS Managed ElasticSearch and is only available over the REST endpoint. Consider switching your client to consume the REST endpoint, such as https://github.com/searchbox-io/Jest .

Source: https://forums.aws.amazon.com/thread.jspa?messageID=681938

Answer 2

Since the Elasticsearch Java SDK version 5.6 there is a REST Client available. This allows you to connect to Elasticsearch Service on AWS.

Currently Elasticsearch Service allows installations up to version 5.5, but you can use the 5.6 Java SDK against a 5.5 cluster with minor limitations.

Note : When initializing the RestClient you should use the port 80 or 443 respectively instead of the 9200 . Eg

RestClient restClient = RestClient.builder(
    new HttpHost("search-test-elasti-xxxx-xxxxx.us-east-1.es.amazonaws.com", 80, "http")).build();
RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClient); 
// [...]

Answer 3

How to get the AWS ES URL

Go to the Elasticsearch Domain console and get the Endpoint in the Overview tab.

Port to use is 443.

Make sure the access control has been configured.

• Options for granting or denying access to Amazon ES endpoints

---

FYI

Create an Elasticsearch Index

curl -v -XPUT "${ES_ENDPOINT}/article/" -H 'Content-Type: application/json' -d '{
    "settings" : {
        "index" : {
            "number_of_shards" : 1,
            "number_of_replicas" : 0
        }
    }
}'

Answer 4

As John Russell said above, you need to use a REST client to communicate with your AWS Elastic cluster.

Elastic recently released the first RC version of its own Java REST client, so this is an option now as well.

Client Docs: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html

Maven Repo: http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.elasticsearch.client%22%20AND%20a%3A%22rest%22