Why is Python raising exception, despite a bare Except block?

Question

Imagine the following code:

def query(sql, data):
    with conn as cursor:
        try:
            cursor.execute(sql, data)
            rows = cursor.fetchall()
            conn.commit()
        except Exception as e:
            print(cursor._last_executed)
            print(e)

When calling it, an err.InterfaceError("(0, '')") is risen from the last line: print(e) .

I could even understand it if it was risen from print(cursor._last_executed) , as cursor might be not available or something. But this is not the case.

Why, when my bare exception is supposed to handle everything?

Please ignore the discussion whether bare exceptions are a good or bad practice, this is another topic. The question is how come the exception is risen at all in this case.

Edit : the exception is risen very rarely, under heavy DB load. You will not be able to reproduce it.

Edit2 : I managed to copy the traceback as text from the sentry report:

InterfaceError: (0, '')
  File "run_signal_generator.py", line 39, in <module>
    main()
  File "run_signal_generator.py", line 35, in main
    ds.run_trades_stream()
  File "/home/deribit/rubber-band/data_stream/data_streamer.py", line 223, in run_trades_stream
    self.process_data(data)
  File "/home/deribit/rubber-band/data_stream/data_streamer.py", line 97, in process_data
    self.start_new_candle(timeframe)
  File "/home/deribit/rubber-band/data_stream/data_streamer.py", line 117, in start_new_candle
    self.notify()
  File "/home/deribit/rubber-band/data_stream/observer.py", line 13, in notify
    observer.update()
  File "/home/deribit/rubber-band/data_stream/observer.py", line 26, in update
    self.process_data()
  File "/home/deribit/rubber-band/data_stream/signal_generator.py", line 131, in process_data
    return self.process_potential_new_trade()
  File "/home/deribit/rubber-band/data_stream/signal_generator.py", line 160, in process_potential_new_trade
    return self.process_enter_signal()
  File "/home/deribit/rubber-band/data_stream/signal_generator.py", line 407, in process_enter_signal
    trade_id = self.store_enter_signal_db(data)
  File "/home/deribit/rubber-band/data_stream/signal_generator.py", line 522, in store_enter_signal_db
    return query(sql, db_data)["id"]
  File "/home/deribit/rubber-band/helpers/mysql.py", line 19, in query
    print(e)
  File "pymysql/connections.py", line 881, in __exit__
    self.commit()
  File "pymysql/connections.py", line 798, in commit
    self._execute_command(COMMAND.COM_QUERY, "COMMIT")
  File "pymysql/connections.py", line 1122, in _execute_command
    raise err.InterfaceError("(0, '')")

Indeed it claims the exception is risen from this line.

Answer 1

Seems similar to this old question : mysql connections and cursors are not thread-safe so if you're sharing connections between multiple threads (instead of having one connection per thread) their state can break at essentially any moment.

It's possible that for some reason __str__ on MySQL's connection errors ping the server to try and get more information, which break if eg an other thread has already changed the connection state. cursor._last_executed might not have that issue due to just saving the information locally either when executing the request or when the response comes back.

And as other commenters noted, if you get an exception provide the entire traceback and error message not a small fraction of the information. If you need to anonymise part of the traceback do so but the less ESP or assumptions are necessary the better.

Answer 2

Figured it.

This is what is going on:

1) The main query raise an error.

2) It is handled.

3) As the query is inside a context manager, the context manager exites it's object, eg connection instance.

4) As the MySQL server has gone away, the __exit__ method of the connection class can not execute properly.

5) As it can not execute properly, it raises an error within the scope of the context manager , outside the scope of the bare handling exception inside query .

You can get the same result if you experiment with the following code:

class ContextManagerException(Exception):
    pass


class TryBlockException(Exception):
    pass


class A(object):
    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, tb):
        raise ContextManagerException


with A() as a:
    try:
        raise TryBlockException
        pass
    except Exception as e:
        print(e)

As you can see, the risen exception is indeed risen from the context manager, and indeed it is risen from the print(e) line, as it is the last line to execute from the context manager.