Using AWS Secrets Manager with Python (Lambda Console)

Question

I am attempting to use Secrets Manager a Lambda function in AWS. Secrets a manager is used to store database credentials to Snowflake (username, password).

I managed to set up a secret in Secrets Manager which contains several key/value pairs (eg one for username, another for password).

Now I am trying to refer to these values in my Python function code. AWS documentation kindly provides the following snippet:

import boto3
import base64
from botocore.exceptions import ClientError


def get_secret():

    secret_name = "MY/SECRET/NAME"
    region_name = "us-west-2"

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    # In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
    # See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
    # We rethrow the exception by default.

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        if e.response['Error']['Code'] == 'DecryptionFailureException':
            # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InternalServiceErrorException':
            # An error occurred on the server side.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            # You provided an invalid value for a parameter.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            # You provided a parameter value that is not valid for the current state of the resource.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'ResourceNotFoundException':
            # We can't find the resource that you asked for.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
    else:
        # Decrypts secret using the associated KMS CMK.
        # Depending on whether the secret is a string or binary, one of these fields will be populated.
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])

    # Your code goes here.

Later in my def lambda_handler(event, context) function, I have the following snippet to establish a connection to my database:

        conn = snowflake.connector.connect(
            user=USERNAME,
            password=PASSWORD,
            account=ACCOUNT,
            warehouse=WAREHOUSE,
            role=ROLE
            )

However, I am unable to figure out how to use the get_secret() function to return values for parameters like USERNAME or PASSWORD .

How can this be accomplished? Thank you for the help!

Answer 1

update the last part of get_secret() to:

else:
        # Decrypts secret using the associated KMS CMK.
        # Depending on whether the secret is a string or binary, one of these fields will be populated.
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            secret = base64.b64decode(get_secret_value_response['SecretBinary'])

return json.loads(secret)  # returns the secret as dictionary

This will return a dictionary where you'll have the keys you specified in AWS Secret Manager console.

Answer 2

• Here is how i have used it using arn, following this bloc hope that helps you.
• Worth checking what you have used to store and accordingly use one SecretString or SecretBinary

    secrets_client = boto3.client('secretsmanager')
    secret_arn = 'arn:aws:secretsmanager:eu-west-2:xxxxxxxxxxxx:secret:dashboard/auth_token'
    auth_token = secrets_client.get_secret_value(SecretId=secret_arn).get('SecretString')

• boto3 docs
• get_secret_value Retrieves the contents of the encrypted fields SecretString or SecretBinary from the specified version of a secret, whichever contains content.
• Your lambda role should have the following permissions depending on what is used
  • secretsmanager:GetSecretValue
  • kms:Decrypt required only if you use a customer-managed AWS KMS key to encrypt the secret. You do not need this permission to use the account's default AWS managed CMK for Secrets Manager.

Answer 3

hi @Prashanth kumar did u mean something as this: def get_secret():

secret_name = "MySecretForRedshift"
region_name = "us-east-1"

# Create a Secrets Manager client
session = boto3.session.Session()
client = session.client(
    service_name='secretsmanager',
    region_name=region_name
)

# In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
# See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
# We rethrow the exception by default.

try:
    get_secret_value_response = client.get_secret_value(
        SecretId=secret_name
    )
except ClientError as e:
    if e.response['Error']['Code'] == 'DecryptionFailureException':
        # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
        # Deal with the exception here, and/or rethrow at your discretion.
        raise e
    elif e.response['Error']['Code'] == 'InternalServiceErrorException':
        # An error occurred on the server side.
        # Deal with the exception here, and/or rethrow at your discretion.
        raise e
    elif e.response['Error']['Code'] == 'InvalidParameterException':
        # You provided an invalid value for a parameter.
        # Deal with the exception here, and/or rethrow at your discretion.
        raise e
    elif e.response['Error']['Code'] == 'InvalidRequestException':
        # You provided a parameter value that is not valid for the current state of the resource.
        # Deal with the exception here, and/or rethrow at your discretion.
        raise e
    elif e.response['Error']['Code'] == 'ResourceNotFoundException':
        # We can't find the resource that you asked for.
        # Deal with the exception here, and/or rethrow at your discretion.
        raise e
else:
    # Decrypts secret using the associated KMS CMK.
    # Depending on whether the secret is a string or binary, one of these fields will be populated.
    if 'SecretString' in get_secret_value_response:
        secret = json.loads(get_secret_value_response['SecretString'])
    else:
        secret = json.loads(base64.b64decode(get_secret_value_response['SecretBinary']))
return secret

Answer 4

I created an open source library called pysecret, here's the documentation of AWS Secret Manager integration: https://github.com/MacHu-GWU/pysecret-project#aws-key-management-service-and-secret-manager-integration

I can walk you through the easiest way to do that:

1. manually put your secret value in json or create one with pysecret .

from pysecret import AWSSecret

aws_profile = "my_aws_profile"
aws = AWSSecret(profile_name=aws_profile)

secret_id = "my-example-secret"
secret_data = {
    "host": "www.example.com",
    "port": 1234,
    "database": "mydatabase",
    "username": "admin",
    "password": "mypassword",
    "metadata": {
        "creator": "Alice"
    }
}
aws.deploy_secret(name=secret_id, secret_data=secret_data) # or you can pass kms_key_id if you created a custom kms key

Then you should be able to see the secret been created in your aws console.

2. read your secret value in lambda function or in any of your python code .

aws = AWSSecret(profile_name=aws_profile) # in lambda code, don't need ``profile_name=aws_profile``
password = aws.get_secret_value(secret_id="my-example-secret", key="password") # mypassword
creator = aws.get_secret_value(secret_id="my-example-secret", key="metadata.creator") # Alice

NOTE, Lambda Function IAM Role Requirement to access the secret

1. your Lambda IAM role has to have Secret Manager read access. the aws built-in policy arn:aws:iam::aws:policy/SecretsManagerReadWrite has both read and write, if you are lazy, you can use it. But I recommend you to create a custom policy only has the Read access.
2. if you use auto-generated kms key for your secret, you have to use aws kms create-grant command to grant the Lambda Function IAM Role to access the kms key for encryption, here's how https://docs.aws.amazon.com/cli/latest/reference/kms/create-grant.html
3. if you use custom kms key, you should be able to edit the user of your kms key, select the Lambda Function IAM Role in your aws console.

Hope this answers your question.

If this help, please Star my project https://github.com/MacHu-GWU/pysecret-project .

Answer 5

AWS provide some template for some supported database engines such as MySQL etc, please take a look at this template :

For unsupported db, check this

The template provided above will give you an example for customizing for your own function.