如何使用 Lambda 函数将下载的文件上传到 s3 存储桶

Question

I saw different questions/answers but I could not find the one that worked for me. Hence, I am really new to AWS, I need your help. I am trying to download gzip file and load it to the json file then upload it to the S3 bucket using Lambda function. I wrote the code to download the file and convert it to json but having problem while uploading it to the s3 bucket. Assume that file is ready as x.json. What should I do then?

I know it is really basic question but still help needed :)

Answer 1

This code will upload to Amazon S3:

import boto3

s3_client = boto3.client('s3', region_name='us-west-2')  # Change as appropriate
s3._client.upload_file('/tmp/foo.json', 'my-bucket', 'folder/foo.json')

Some tips:

• In Lambda functions you can only write to /tmp/
• There is a limit of 512MB
• At the end of your function, delete the files (zip, json, etc) because the container can be reused and you don't want to run out of disk space

Answer 2

If your lambda has proper permission to write a file into S3, then simply use boto3 package which is an AWS SDK for python.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html

Be aware that if the lambda locates inside of VPC then lambda cannot access to the public internet, and also boto3 API endpoint. Thus, you may require a NAT gateway to proxy lambda to the public.