Connect Redshift and Python without using aws credentials (running on emr)

Question

Hi I am new to AWS world, I need help in connecting Python to Redshift Database. How can I do it without giving the credentials as I am running it on the EMR provided I am not using any third party services

Answer 1

Amazon EMR runs on Amazon EC2 instances. These instances are assigned an IAM Role, which grants permission to software running on the instances when they use an AWS SDK.

For example, when software running on EMR access S3 it obtains credentials via EC2 instance metadata , which includes a set of temporary credentials associated with the IAM Role assigned to the instance.

You can call the Amazon Redshift Data API in a similar manner and it will use credentials provided through the IAM Role.

You are using Python, so you can use the boto3 RedshiftDataAPIService — Boto3 documentation to query an Amazon Redshift cluster using permission supplied by the IAM Role.

You would need to add appropriate permissions to the IAM Role being used by the EMR cluster. See: Configure IAM service roles for Amazon EMR permissions to AWS services and resources - Amazon EMR