cron 未启动有效的 python 脚本

Question

I have more than some troubles getting cron working on my raspberry pi 3b. I wrote a python script and I have no issues launching it from thonny or terminal, it runs smoothly.

However, when launching the cron job in the following way (changing minutes and hours accordingly to test time):

25 21 * * * /usr/bin/python3 /home/pi/main.py >> /home/log/myscript.log 2>&1

I get the following error message:

Traceback (more recent call last):
   File "/home/pi/main.py", line 9, in <module>
       import tweepy
ImportError: No module named 'tweepy'

No matter if I call sudo crontab -e , nano crontab -e or cron -e . I started investigating and I guessed that there where problems with the python version to call.

Therefore I substituted the python3 call with python3.5 , later with python3.5m and even with python3.8 with no results. In any attempt I tried launching the same command via terminal and it worked fine. Tried to unistall and reinstall tweepy and checked the python used by Thonny which is the 3.5 version.

What can be trouble? It's driving me mad.

EDIT: I backed up my scripts and re-install raspberry pi to the latest version (5.10.17 linux kernel). Again, I tried using the script with both Thonny and terminal and it run.

Again, I had the same problem using crontab.

However, I had an idea. I updated the tweepy version to the latest version available, it was like a desperate attempt and something changed.

Instead of having the error related to tweepy, cron gives me this:

Traceback (most recent call last):
  File "/home/pi/Documents/TwitterBot/main.py", line 17, in <module>
    a = soup.select('table > tr:has(> td > a:-soup-contains("San Pellegrino Terme")) td')
  File "/usr/lib/python3/dist-packages/bs4/element.py", line 1376, in select
    return soupsieve.select(selector, self, namespaces, limit, **kwargs)
  File "/usr/lib/python3/dist-packages/soupsieve/__init__.py", line 112, in select
    return compile(select, namespaces, flags, **kwargs).select(tag, limit)
  File "/usr/lib/python3/dist-packages/soupsieve/__init__.py", line 63, in compile
    return cp._cached_css_compile(pattern, namespaces, custom, flags)
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 205, in _cached_css_compile
    CSSParser(pattern, custom=custom_selectors, flags=flags).process_selectors(),
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 1010, in process_selectors
    return self.parse_selectors(self.selector_iter(self.pattern), index, flags)
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 852, in parse_selectors
    has_selector, is_html = self.parse_pseudo_class(sel, m, has_selector, iselector, is_html)
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 510, in parse_pseudo_class
    has_selector = self.parse_pseudo_open(sel, pseudo, has_selector, iselector, m.end(0))
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 654, in parse_pseudo_open
    sel.selectors.append(self.parse_selectors(iselector, index, flags))
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 852, in parse_selectors
    has_selector, is_html = self.parse_pseudo_class(sel, m, has_selector, iselector, is_html)
  File "/usr/lib/python3/dist-packages/soupsieve/css_parser.py", line 585, in parse_pseudo_class
    "'{}' pseudo-class is not implemented at this time".format(pseudo)
NotImplementedError: ':-soup-contains' pseudo-class is not implemented at this time

Which is a trouble related to the version of beautifulsup Cron is running.

I tried to update beautifulsup to the latest version available with no results.

I am starting to run out of idea, Any help would be appreciated.

Answer 1

If you are running a virtual environment your paths in a shell script need to be for that virtual environment. Here's how I run shell on AWS linux instance. You can play around with this but it took me a while to figure it out. Maybe just add the long environment path in cron and skip the shell script.

This is inside my_shell.sh

/usr/local/bin/pipenv run ~/.local/share/virtualenvs/ec2-user-zzkNbF-x/bin/python /home/ec2-user/my_path_to_file/my_file.py

This is my crontab setting

0 21 * * mon-fri /home/ec2-user/my_path_to_file/my_shell.sh