AWS Elastic Kubernetes Service: how to expose a container/pod to an Elasticsearch cluster inside AWS?

Question

Setup: there is an EKS cluster running with 2 worker nodes and there is a separate Elasticsearch cluster as well in the same VPC as the worker nodes. How can I / shall I open a connection between a logstash container in a pod on a worker node to the Elasticsearch cluster? I guess a service is needed for logstash but what type and how to set it up? Thanks for answering!

Answer 1

As a comment alluded to, you can do this via standard AWS Security Group adjustments. ie- Make sure that your worker nodes' security group allows Outbound connectivity to your Elasticsearch cluster on port 9200 or whatever port you're using, and make sure that your Elasticsearch Cluster Ec2 instances allow Inbound traffic from your Worker Nodes on port 9200. This assumes you're not using AWS's nifty new Security Group per Pod functionality, which allows you to get even more granular with your rules.

And then to test, you can exec into your logstash pod and curl your elasticsearch cluster endpoint. You can install curl if it's not already installed.

kubectl exec -it <logstash-pod> /bin/bash

curl -XGET <elasticsearch-url>/_cluster/health