What is the best way to run python scripts once per month in AWS?

Question

I need to run a python script file on the 20th of every month. This code will read the tsv file (stored somewhere) then call to the fee registration API and log the response to a file and save it. The processed tsv file will be moved to the processed folder. I dont know the best way to implement this in AWS?

Manual approach is:

• Create EC2 instance, run python script
• Store tsv file on s3 (I dont know the best way to handle file upload problem)
• Use lambda to trigger on the 20th of every month.

What is the best way to implement this in AWS? What is the best way to implement file storage and upload?

Answer 1

You can do it without an Amazon EC2 instance:

• Create an AWS Lambda function that will perform the process (call API and log the response)
• Store any persistent data in an Amazon S3 bucket and have the Lambda function download/upload from there
• Create a CloudWatch Events rule to trigger the AWS Lambda function at the desired interval

AWS Lambda functions provide 512MB of storage in the /tmp/ directory, so you can download the TSV file to there, perform your operations and then upload the resulting file back to S3.

You will only be charged for the duration that the Lambda function actually runs.