如何使用 lambda 函数和 boto3 从 s3 存储桶中读取 csv 文件？

Question

I have s3 bucket and I have set a lambda function which will display contents of csv file when a csv file is uploaded to s3 bucket.S3 bucket is already set as a trigger for my lambda function. Can you please suggest and advise?

Answer 1

An AWS Lambda function is code that you write. You can make it do anything you wish.

For your first scenario of displaying a CSV file in CloudWatch Logs, the Lambda function should:

• Retrieve the name of the bucket and object from the event passed to the Lambda function
• Download the file from Amazon S3 to the /tmp/ directory
• Read the CSV using normal Python code and print() the information that you wish to appear in CloudWatch Logs
• Delete the temporary file, so as not to consume too much disk space (there is a limit of 500MB of temporary disk space, and Lambda containers can be reused multiple times)

For your second question of "adding an extra column", the Lambda function should:

• Retrieve the name of the bucket and object from the event passed to the Lambda function
• Download the file from Amazon S3 to the /tmp/ directory
• Manipulate the contents of the file however you wish, using Python code
• Upload the file to Amazon S3
• Delete the temporary file, so as not to consume too much disk space (there is a limit of 500MB of temporary disk space, and Lambda containers can be reused multiple times)

The code would look something like:

import urllib
import boto3

# Connect to S3 and DynamoDB
s3_client = boto3.client('s3')

def lambda_handler(event, context):

    # Get the bucket and object key from the Event
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'])
    localFilename = '/tmp/file.txt'

    # Download the file from S3 to the local filesystem
    s3_client.download_file(bucket, key, localFilename)

    # Do stuff here with the local file (your code here!)
    pass

    # Upload modified file
    s3_client.upload_file(localFilename, bucket, key)