I have a python script that connects to Redshift, executes a series of SQL commands, and generates a new derived table.
But for the life of me, I can't figure out a way to have it automatically run every day.
I've tried AWS Data Pipeline but my shell script won't run the first copy statement. I can't get Lambda or Glue to work because my company's IAM policies are restrictive. Airflow seems like overkill to just run a single python script daily.
Any suggestions for services to look into?
Cron job?
00 12 * * * /home/scottie/bin/my_python_script.py
Run my_python_script.py
at the top of the hour (0th minute), at noon, every day.
I use a scheduled task on Windows. Either enter it using the GUI or the at command .
If you are using AWS Glue or have some other reason to install a development endpoint, you can use Apache Zeppelin to run any code from any language (if you have the jar files) on a schedule based on a cron command. Here's the notebook I use to run Redshift nightly maintenance:
在ec2实例上使用cron作业或设置调度事件来调用你的aws python lambda函数http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.