Unable to connect to aws redshift from python within lambda

Question

I am trying to connect to redshift with python through lambda. The purpose is to perform queries on the redshift database.

I've tried this by getting the temp aws credentials and connecting with psycopg2, but it isn't successful without any error messages. (IE: the lambda just time out)

rs_host = "mytest-cluster.fooooooobaaarrrr.region111111.redshift.amazonaws.com"
rs_port = 5439
rs_dbname = "dev"
db_user = "barrr_user"

def lambda_handler(events, contx):
    # The cluster_creds is able to be obtained successfully. No issses here
    cluster_creds = client.get_cluster_credentials(DbUser=db_user,
                                                   DbName=rs_dbname,
                                                   ClusterIdentifier="mytest-cluster",
                                                   AutoCreate=False)

    try:
        # It is this psycopg2 connection that cant work... 
        conn = psycopg2.connect(host=rs_host,
                                port=rs_port,
                                user=cluster_creds['DbUser'],
                                password=cluster_creds['DbPassword'],
                                database=rs_dbname
    )
        return conn

    except Exception as e:
        print(e)

Also, the lambda execution role itself has these policies:

I am not sure why am I still not able to connect to redshift via python to perform queries.

I have also tried with the sqlalchemy libary but no luck there.

Answer 1

As what Johnathan Jacobson mentioned above. It was the security groups and network permissions that caused my problem.

Answer 2

You can maybe review the documentation at Create AWS Lambda Function to Connect Amazon Redshift with C-Sharp in Visual Studio

Since you have already your code in Python, you can concentrate on the networking part of the tutorial

While launching AWS Lambda functions, it is possible to select a VPC and subnet where the serverless lambda function servers will spinup

You can choose exactly the same VPC and the subnet(s) where you have created your Amazon Redshift cluster

Also, revise the IAM role you have attached to the AWS Lambda function. It requires additionally the AWSLambdaVPCAccessExecutionRole policy

This will be solving issues between connections from different VPCs Again, even you have launched the lambda function in the same VPC and subnet with Redshift cluster, it is better to check the security group of the cluster so that it accepts connections

Hope it works,