Python sudo cron job not working due to import module error

Question

I have a web scraper script which runs without any problem when I start via command line. I have created a crontab job to run this script periodically with:

sudo crontab -e

According to syslog message script started but there was no expected output, so I created a log file for stdout and stderr messages and there seems to be a problem with BeautifulSoup module import.

import BeautifulSoup
ImportError: No module named bs4

Do you have any idea what might be the root cause and how I can solve it?

EDIT: My problem was related with my crontabs run by different user (root). So as suggested in the answer, when I changed it to own user, problem solved!

Answer 1

the sudo infront of crontab means that root user runs the script. Beautiful soup is not installed for root user so it cannot find the module.

Instead remove the statement from "sudo crontab -e" and then run and just "crontab -e". Placing the script in there will run it from your user account and should have access to the beautiful soup module

Answer 2

for every user which you use for cron jobs, you should do pip install your-library . Because cron uses the environment whose user run the job.

Answer 3

If it's necessary to run a Python script as sudo in crontab, you should utilize the PYTHONPATH variable (as mentioned in a comment by Sraw) to augment the default search path for module files.

First, determine the location of the module by, eg running your Python (where the module is imported successfully) and checking sys.path :

import sys
print(sys.path)

One of the paths listed will be the location (call it location/of/module ). Find it and change your entry in sudo crontab -e to

PYTHONPATH=/location/of/module python your_script.py