亚马逊AWS elasticsearch从浏览器访问Kibana

Question

I know this issue has been already discussed before , Yet I feel my question is a bit different.

I'm trying to figure out how am I to enable access to the Kibana over the self manged AWS elastic search which I have in my AWS account .

Could be that what am I about to say here is inaccurate or complete nonsense . I am pretty novice in the whole AWS VPC wise section and to ELK stuck.

Architecture: Here is the "Architecture":

1. I have a VPC.
2. Within the VPC I have several sub nets.
3. Each server sends it's data to the elastic search using log stash which runs on the server itself. For simplicity lets assume I have a single server.
4. The elastic search https url which can be found in the Amazon console is resolved to an internal IP within the sub net that I have defined.

Resources:

I have found the following link which suggest to use one of two option:

https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/

Solutions:

Option 1: resource based policy

1. Either to allow resource based policy for elastic search by introducing condition which specify certain IP address.

This was discussed in the following thread but unfortunately did not work for me.

Proper access policy for Amazon Elastic Search Cluster

When I try to implement it in the Amazon console, Amazon notifies me that because I'm using Security group , I should resolve it by using security group.

Security group rules:

I tried to set a rule which allows my personal computer(Router) public IP to access Amazon elastic search ports or even opening all ports to my public IP.

But that didn't worked out. I would be happy to get a more detailed explanation to why but I'm guessing that's because the elastic search has only internal IP and not public IP and because it is encapsulated within the VPC I am unable to access it from outside even if I define a rule for a public IP to access it.

Option 2: Using proxy

I'm decline to use this solution unless I have no other choice.

I'm guessing that if I set another server with public and internal IP within the same subnet and VPC as that of the elastic search , and use it as a proxy, I would be then be able to access this server from the outside by defining the same rules to the it's newly created security group . Like the article suggested.

Sources:

I found out of the box solution that some one already made for this issue using proxy server in the following link:

Using either executable or docker container.

https://github.com/abutaha/aws-es-proxy

Option 3: Other

Can you suggest other solution? Is it possible to use Amazon Load balancer or Amazon API gateway to accomplish this task?

I just need proof of concept not something which goes into production environment.

Bottom line:

I need to be able to aceess Kibana from browser in order to be able to search elastic search indexes.

Thanks a lot

Answer 1

The best way is with the just released Cognito authentication.

https://aws.amazon.com/about-aws/whats-new/2018/04/amazon-elasticsearch-service-simplifies-user-authentication-and-access-for-kibana-with-amazon-cognito/

This is a great way to authenticated A SINGLE USER . This is not a good way for the system you're building to access ElasticSearch.