How to run python scripts ? Also need to know how to automate the scripts

Question

I have written 3 python scripts. One script has mysql database connection (say name db.py). One script contains n no of parameterized methods (say name analytics.py). Now the data is fetched into analytics.py and converted into dataframe using "import db" in analytics.py. Just so you know, in analytics.py the methods are only defined. The methods from this script are called from another python script (say name main.py). In main.py, I have passed the parameters as command line arguments. Till now I am running the python script using the shell script I have written. Once I run the.sh file and give inputs to that.sh file, some file are expected to be generated at a particular location. Also running the script is done manually for now.

I want to know a way or a few, by which I can automate this?

Also if you could give me any suggestions on running the python script in some other way would be appreciated.

Please help me with the process of what to do after writing the python scripts.

Thanks.

Answer 1

Based on the fact that you are using ".sh" files, I will assume you're using Linux, in that case you can use crontab to run commands based on a time schedule.

Answer 2

It's complicated to answer that question without actually looking at the code. However, I may be able to share some tools which will make your life easier.

To begin with, there is a library call "begin" that is really powerful. You can read more about it here: https://pypi.org/project/begins/

This is an example:

import begin

@begin.start
def run(color = 'blue', height = 100, width = 200, path = '/tmp/out.jpg'):
    print('Color:', color)
    print('Resolution:', width, height)
    print('Image:', path)

After that you can run the script by simply running:

>>> python3 script.py --color 'red' --height 400 --path /var/tmp/test.png
Color: red
Resolution: 200 400
Image: /var/tmp/test.png

To automate the execution of the script, you can use cron, as described here: https://opensource.com/article/17/11/how-use-cron-linux

Basically, if you are on Linux, you can edit your crontab by running:

crontab -e

Then, you can insert the following line:

* */12 * * * python3 $HOME/script.py --color 'red' --height 400 --path /var/tmp/test.png > /tmp/script.log

That will run your script every 12 hours and send output to /tmp/script.log .

You can use this crontab generator to generate that cron line: https://crontab-generator.org/