Kibana 4.6.x not starting on Google compute Engine

Question

I am using ELK stack on GCE. Everything was running properly. But then I installed NGINX and later uninstalled it. After that Kibana is not able to start after unistalling NGINX.

Elasticsearch Version: 2.4.5 Kibana Version: 4.6

Kibana STDOUT logs are:

{"type":"log","@timestamp":"2017-06-06T08:13:06Z","tags":["status","plugin:kibana@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:elasticsearch@1.0.0","info"],"pid":364,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:kbn_vislib_vis_types@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:markdown_vis@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uni nitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:metric_vis@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:spyModes@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:elasticsearch@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from yellow to green - Kibana index ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:statusPage@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitializ ed"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["status","plugin:table_vis@1.0.0","info"],"pid":364,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"} {"type":"log","@timestamp":"2017-06-06T08:13:07Z","tags":["fatal"],"pid":364,"level":"fatal","message":"listen EADDRNOTAVAIL 104.155.222.239:5601","error":{"message":"listen EADDRNOTAVAIL 104.155.222.239:5601","name":"Error","stack":"Error: listen EADDRNOTAVAIL 104.155.222.239:5601\\n at Object.exports._errnoException (util.js:1026:11)\\n at exports._exceptionWithHostPort (util.js:1049:20)\\n at Server._listen2 (net.js:1244:19)\\n at listen (net.js:1293:10)\\n at net.js:1403:9\\n at _combinedTickCallback (internal/process/next_tick.js:77:11)\\n at process._tickCallback (internal/process/next_tick.js:98:9)","code":"EADDRNOTAVAIL"}}

Nothing is running on port 5601 and it is allowed by FIREWALL.

Answer 1

Based on the last few messages in the log, it looks like Kibana is trying to listen on the following IP/PORT combination 104.155.222.239:5601 and that is the reason for the EADDRNOTAVAIL error code.

Since you've provided very minimal information, these are things you could verify:

1. Verify that the IP address 104.155.222.239 is the IP address of the instance. This looks like a public IP address. If you're using ephemeral public (ie external) IP addresses, note that the address might change between instance restarts. You might have to get a static IP address from Google Compute Engine to avoid this problem or determine the external IP address correctly and pass it to your program.

2. Confirm there are no processes listening on port 5601 .

You can run the following command to print all processes listening on that port:

lsof -i:26625