Kibana将无法通过Amazon的Elasticsearch Service连接到Elasticsearch

Question

After switching from hosting my own Elastiscsearch cluster to Amazon's Elasticsearch Service , my Kibana dashboards (versions 4.0.2 and 4.1.2) won't load and I'm receiving the following error in kibana.log:

{
  "name": "Kibana",
  "hostname": "logs.example.co",
  "pid": 8037,
  "level": 60,
  "err": {
    "message": "Not Found",
    "name": "Error",
    "stack": "Error: Not Found\n    at respond (\/srv\/kibana\/kibana-4.1.2-linux-x64\/src\/node_modules\/elasticsearch\/src\/lib\/transport.js:235:15)\n    at checkRespForFailure (\/srv\/kibana\/kibana-4.1.2-linux-x64\/src\/node_modules\/elasticsearch\/src\/lib\/transport.js:203:7)\n    at HttpConnector.<anonymous> (\/srv\/kibana\/kibana-4.1.2-linux-x64\/src\/node_modules\/elasticsearch\/src\/lib\/connectors\/http.js:156:7)\n    at IncomingMessage.bound (\/srv\/kibana\/kibana-4.1.2-linux-x64\/src\/node_modules\/elasticsearch\/node_modules\/lodash-node\/modern\/internals\/baseBind.js:56:17)\n    at IncomingMessage.emit (events.js:117:20)\n    at _stream_readable.js:944:16\n    at process._tickCallback (node.js:442:13)"
  },
  "msg": "",
  "time": "2015-10-14T20:48:40.169Z",
  "v": 0
}

unfortunately, this error is not very helpful. I assume it's a wrapped HTTP 404, but for what ?

How can I connect a Kibana install to Amazon's Elasticsearch Service?

Answer 1

Here are a some things to keep in mind when using Amazon's Elasticsearch Service:

1. Modifications to the access policies take a non-deterministic amount of time. I've found that it's good to wait at least 15 minutes after the status is no longer "processing" after making policy changes.

2. It listens on port 80 for HTTP requests and not the standard port 9200. Be sure that your elasticsearch_url configuration directive reflects this, eg:

   elasticsearch_url: " http://es.example.co:80 "

3. It's very likely that your Kibana instance will not have the necessary permissions to create the index that it needs to show you a dashboard -- this is towards the root of the issue. Check out the indexes on the Elasticsearch domain and look for a line that matches the kibana_index config directive (eg via http://es.example.co/_cat/indices ).

for instance, if your kibana_index directive is the value is .kibana-4 , if you don't see a line like the following:

green open .kibana-4           1 1      3 2  30.3kb  17.2kb

then your Kibana index was not able to create the index it needs. If you go to the dashboard for the Elasticsearch service in amazon and click on the Kibana link, it will likely create a .kibana-4 index for you.

You can specify this index in your existing Kibana's configuration and you should see the next point.

4. Your existing Kibana install will likely require authentication via the header:

   Kibana: Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header.

You can configure this in Kibana and can see the general signing API request documentation for more help.

It's worth noting that the error messaging is reportably better in Kibana 4.2, but as that's in beta and Amazon's Elasticsearch Service was very recently released, the above should be helpful in debugging.