Cannot access Kibana remotely on EC2 instance

Question

I'm a newbie to the world of AWS and set up an EC2 instance and installed ElasticSearch to play around a bit with its features and learn more. I have ElasticSearch working: I can send curl requests to the ES service remotely and get back results.

However, I'm unable to access Kibana remotely. I first went to my kibana.yml file and enabled the following settings:

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "127.0.0.1"

Note: for server.host , I've tried 0.0.0.0 , the actual IP address of my EC2 instance, and 127.0.0.1 , and none of them allow my connection access.

I then went to my AWS console and checked to make sure my Security Group had the correct permissions:

I then started my Kibana service:

service kibana start

Within my SSH terminal, I'm able to execute curl localhost:5601 and receive a valid response:

<script>var hashRoute = '/app/kibana';
var defaultRoute = '/app/kibana';

var hash = window.location.hash;
if (hash.length) {
  window.location = hashRoute + hash;
} else {
  window.location = defaultRoute;
} 

However, if I use Postman, my browser, or curl from my local machine to remote access Kibana ( curl XXX.XX.XX.XXX:5601 ) I get a Connection refused error.

I next attempted to inspect my Kibana log using vi /var/log/kibana/kibana.stderr , and found an interesting error message:

    at net.js:1408:9
    at _combinedTickCallback (internal/process/next_tick.js:83:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
  cause:
   { Error: listen EADDRNOTAVAIL XX.XXX.XX.XX:5601
       at Object.exports._errnoException (util.js:1020:11)
       at exports._exceptionWithHostPort (util.js:1043:20)
       at Server._listen2 (net.js:1249:19)
       at listen (net.js:1298:10)
       at net.js:1408:9
       at _combinedTickCallback (internal/process/next_tick.js:83:11)
       at process._tickCallback (internal/process/next_tick.js:104:9)
     code: 'EADDRNOTAVAIL',
     errno: 'EADDRNOTAVAIL',
     syscall: 'listen',
     address: 'XX.XX.XX.XX',
     port: 5601 },
  isOperational: true,
  code: 'EADDRNOTAVAIL',
  errno: 'EADDRNOTAVAIL',
  syscall: 'listen',
  address: 'XX.XXX.XX.XXX',
  port: 5601 }

where the XX.XX... represent my EC2 instance IP address. It seems like something is already bound to port 5601. So I next issued sudo lsof -i:5601 and saw this output:

> COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME node    3676
> kibana   11u  IPv4  14951      0t0  TCP localhost:esmagent (LISTEN)

I'm not at all experienced with Linux, but this appears to say that kibana is correctly bound to this port. So at this point, I'm fresh out of ideas. I've looked through SO posts like this one about the same issue, and have tried their answers (in the workflow above). Anyone have any ideas what's causing this error?

EDIT: I've been able to start Kibana on my local instance fine, and am able to connect to my ElasticSearch service. I just changed the settings in my local kibana.yml file to point to my EC2 instance port 9200. But I'd still love to know why I can't access Kibana on my EC2 instance!

Answer 1

This isn't much of an answer since I'm not sure exactly what fixed the issue, but for future users going through a similar headache, I terminated my instance and reinstalled Java 8, Elastic Search and Kibana entirely. I made sure that ES and Kibana version numbers matched . For example, I was using 5.6 Kibana, but 2.x Elastic Search originally.

Then, make sure that your inbound Security Group settings allow TCP access to ports 9200 and 5601, for ES and Kibana.

Then, as mentioned in the comments, bind Kibana to 0.0.0.0. That's what got mine working.

I know I'm not giving a good theoretical reason for the error, but perhaps this will help check a few things off the list for the next user.

Answer 2

If you are running Kibana as a service, then go to /etc/kibana/kibana.yml and make the

server.host: "0.0.0.0"

and uncomment

server.port
elasticsearch.url

Answer 3

Excerpt from Digital Ocean Blog Install Elastic Stack on centOs7 machine - "Before we can use the Kibana web interface, we have to set up a reverse proxy. Let's do that now, with Nginx.

Install Nginx Because we configured Kibana to listen on localhost, we must set up a reverse proxy to allow external access to it. We will use Nginx for this purpose.

Note: If you already have an Nginx instance that you want to use, feel free to use that instead. Just make sure to configure Kibana so it is reachable by your Nginx server (you probably want to change the host value, in /opt/kibana/config/kibana.yml, to your Kibana server's private IP address). "

Answer 4

对我来说，此问题已通过在Windows防火墙https://www.windowscentral.com/how-open-port-windows-firewall中为端口5601（默认）添加入站规则来解决