如果发生错误，则重新运行 Python 脚本

Question

I am scraping twitter data using their Streaming API, sometimes, I will get

  File "//swigel_local/Shared/Resources/Tools/Twitter/Streaming/StreamingTwitter.py", line 68, in <module>
    stream.filter(track=TRACK_TERMS)

  File "C:\Users\SwigelUser\Anaconda3\lib\site-packages\tweepy\streaming.py", line 445, in filter
    self._start(async)

  File "C:\Users\SwigelUser\Anaconda3\lib\site-packages\tweepy\streaming.py", line 361, in _start
    self._run()

  File "C:\Users\SwigelUser\Anaconda3\lib\site-packages\tweepy\streaming.py", line 294, in _run
    raise exception

  File "C:\Users\SwigelUser\Anaconda3\lib\site-packages\tweepy\streaming.py", line 263, in _run
    self._read_loop(resp)

  File "C:\Users\SwigelUser\Anaconda3\lib\site-packages\tweepy\streaming.py", line 313, in _read_loop
    line = buf.read_line().strip()

AttributeError: 'NoneType' object has no attribute 'strip'

And this will stop my program. Is there a way that I can rerun my program automatically when this error occurs?

My code is :

class StreamListener(tweepy.StreamListener):

    def on_status(self, status):
        if status.retweeted:
            return

        text = status.text
        created = status.created_at
        id_str = status.id_str
        retweets = status.retweet_count
        blob = TextBlob(text)
        sent = blob.sentiment

        table = db[settings.TABLE_NAME]
        try:
            table.insert(dict(
                text = text,
                created = created,
                id_str = id_str,
                retweet_count = retweets,
                polarity = sent.polarity,
                subjectivity = sent.subjectivity,
            ))
        except ProgrammingError as err:
            print(err)


    def on_error(self, status_code):
        if status_code == 420:
            #returning False in on_data disconnects the stream
            return False
stream.filter(track=TRACK_TERMS)

I don't know why the error happens. But I think it has something to do with the change I made recently.

Before, my table in MySQL has utf8 collation, but this will cause warning, then I read articles, and changed my table to utf8mb4 unicode. The warning disappeared and the program runs fine. But sometimes, the above error would occur when the program is running.

def _read_loop(self, resp):
    charset = resp.headers.get('content-type', default='')
    enc_search = re.search('charset=(?P<enc>\S*)', charset)
    if enc_search is not None:
        encoding = enc_search.group('enc')
    else:
        encoding = 'utf-8'

    buf = ReadBuffer(resp.raw, self.chunk_size, encoding=encoding)

    while self.running and not resp.raw.closed:
        length = 0
        while not resp.raw.closed:
            line = buf.read_line().strip()
            if not line:
                self.listener.keep_alive()  # keep-alive new lines are expected
            elif line.isdigit():
                length = int(line)
                break
            else:
                raise TweepError('Expecting length, unexpected value found')

Answer 1

Is there a way that I can rerun my program automatically when this error occurs?

You could write an external application that monitors the system, detects crashes of your application and restarts it.

BUT I wouldn't recommend it!

The right solution is:

• Identify the cause of this issue and fix it!
• Wrap problematic code segments using try .. except , catch such errors (eg AttributeError ) and handle them accordingly