如何调试由 crontab 执行的 python 脚本？

Question

Currently I have a python script that has the following structure

read_data()
    print("debugging statement 1")
    # random code 

plot_data(data)
    print("debugging statement 2")
    #random code 

if __name__ == '__main__':
   var = "test1"
   var2 = "test2"
   data_organized = read_data(var,var2)
   print("debugging statement 3")
   plot_data(data_organized)

   .....rest of code ..... 

I ssh into a raspberry pi and from the terminal I can write:

/full/path/to/env/python3 /full/path/to/script.py 

and the script executes perfectly no errors at all.

I placed it in a crontab to run every minute with the following crontab

*/1 * * * * /full/path/to/env/bin/python3 /full/path/to/script/script.py >> /random/path/to/where/i/want/output/cron_log.txt 2>&1

but the python code throws out an error. when I check my cron_log.txt file I see the error occurred somewhere in the read_data() function because I can see that the print("debugging statement 3) output is off from what it should be. But I cannot see any of the print statements that are inside of the read_data() function.

questions:

1.) why would the code run perfectly when I directly run it but when I run it with crontab it throws out an error?

2.) is my crontab entry correct? I want it to run every minute

3.) how can I write the ouput of the print() functions within the read_data(), plot_Data() functions? currently I can only see print(debugging3) in my output file.

Answer 1

Try with something like this in another script which will call your script.py

#!/usr/bin/env python
# You can name me: logger.py
import subprocess
with open("output.txt", "w+") as output:
    subprocess.call(["python", "./script.py"], stdout=output);

Answer 2

The most likely reason for the error is a permissions issue. (see this https://unix.stackexchange.com/questions/81805/what-are-the-runtime-permissions-of-a-cron-job )

To debug the process, you could use a tool like remote_pdb and then step through your script.

Answer 3

Answering your questions:

1. because cron jobs run in a very different environment compared to your terminal. most likely cron jobs are executed with the different permissions and from a different folder, also you cannot rely on PATH variable being the same (or any other variable for that matter)

2. that seems ok.

3. your script is breaking in read_data() , once you fix this you'll see your debugging output

Finally, why it does not work and how to fix it. Since you haven't provided any meaningful sources, I'd say your read_data() function tries to access some data files using relative paths and fails, since the script is executed from a different folder. You should make sure all paths are absolute and the permissions are set to allow access to those files.