What is the best way to schedule a python script with AWS?

Question

I have a question that I would like to solve:

I have 4 python scripts as follows:

• main.py
  • script1.py
  • script2.py
  • script3.py

Scripts 1, 2 and 3 are run invoked in the main.py.

What I need is to be able to easily schedule this main.py to run once a week.

What AWS services would be best for this? From the architecture side I don't know much.

Thank you!

Answer 1

You can deploy the script in lambda if your execution time is under 15 minutes plus cloudwatch events for scheduling

For execution scripts > 15 minutes, I would suggest using AWS Batch to have the script on schedule on any of the supported compute environment like ECS or EC2

Answer 2

Use Schedule AWS Lambda Functions Using CloudWatch Events

• Create Lambda functions - Deploy these scripts as single lambda function.
• Create EventBridge ridge rule - with lambda as a target Event Source, do the following:Choose Schedule.Choose Fixed rate of and specify the schedule interval (for example, 5 minutes), also as @luk2303 mentioned you can also use cron expressions.
• Use target as lambda created in 1st step.

Use this website for your cron needs https://crontab.guru

Answer 3

For something running once per week, you clearly do not want to have any infrastructure running continuously. This leaves AWS Fargate (good for containerization) or AWS Lambda (good for scripts that run in less than 15 minutes).

Based on the limited information you have provided, AWS Lambda would be a good option since it is simple to schedule.

You would need to change the scripts slightly to fit in the AWS Lambda environment. The Lambda 'function' is triggered via a call to a particular function you nominate. Since you have multiple scripts, you would need to determine whether to combine them all into the one Lambda function, or whether to have them each as a separate function and have the 'main' Lambda function call the other Lambda functions.

AWS Lambda also supports parallel runs by deploying multiple Lambda functions. This could be useful if you have a lot of work to perform, or you can ignore it and just run your code as a 'single thread'.