Elasticsearch Service with Cognito — how to access via Python HTTP request

Question

I recently setup an Elasticsearch service and configured it to use Cognito for identify management. I followed this guide and so far everything is working exactly as expected. I'm able to add new users as expected, and they can access Kibana as expected.

However, I would also like to use Python to interact with the Elasticsearch service. I've following this guide , but I get permission errors about not having the right access.

from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
import boto3

host = 'hostname.us-east-2.es.amazonaws.com/'
region = 'us-east-2'
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)

es = Elasticsearch(
    hosts = [{'host': host, 'port': 443}],
    http_auth = awsauth,
    use_ssl = True,
    verify_certs = True,
    connection_class = RequestsHttpConnection
)

print(es.info())

AuthorizationException: AuthorizationException(403, 'security_exception', 'no permissions for [indices:admin/get] and User [name=arn:aws:iam::12345678:user/username, backend_roles=[], requestedTenant=null]')

I'm not sure if the issue is related to the way that I have Conginto configured, or if it's related to the way that I'm submitting this request. Any help on figuring out where my issue might be would be greatly appreciated.

Answer 1

It happened to me too. There are two ways I found, you have to create an IAM user for both of them.

• Open IAM console and give necessary policies to IAM user for reaching your elasticsearch domain (or I attached "AdministratorAccess" policy only (It gives all access of AWS)), then you should use credentials of this IAM account.
• Or open kibana -> security -> roles -> click all_access role -> mapped_users -> manage_mapping. Then you should add arn of your IAM user to backend roles. You have to use this user's credentials not "boto3.Session().get_credentials()" on your code to connect ES. I suggest the second one if you are not familiar with IAM roles and policies.