pandas.read_sql 'Invalid Cursor state'

Question

i have a flask python app which runs under Apache (using mod_wsgi). A particular URL will cause the app to query a microsoft database using pandas and ceODBC.

ret = pandas.read_sql(sql_str, self._connection)

the sql above is a very basic 'select' on a single table, no joins what so ever. During development this is not a problem (likely because Flask is single threaded). However at run time, the app runs under Apache and i have a particular page that issues 6 simultaneous calls to the same URL. Frequently this causes the following exception:

  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1046, in read_sql
    cursor = self.execute(*args)
  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1041, in execute
    raise_with_traceback(ex)
  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1030, in execute
    cur.execute(*args)
DatabaseError: Execution failed on sql....

And at that time i think the state becomes bad, so subsequent calls will lead to exceptions like these:

File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 421, in read_sql
    coerce_float=coerce_float, parse_dates=parse_dates)
  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1048, in read_sql
    data = self._fetchall_as_list(cursor)
  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1061, in _fetchall_as_list
    result = cur.fetchall()
DatabaseError: [Microsoft][ODBC SQL Server Driver]Invalid cursor state

  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 421, in read_sql
    coerce_float=coerce_float, parse_dates=parse_dates)
  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1048, in read_sql
    data = self._fetchall_as_list(cursor)
  File "C:\proj\pd_test\py_virt_env\Lib\site-packages\pandas\io\sql.py", line 1061, in _fetchall_as_list
    result = cur.fetchall()
DatabaseError: [Microsoft][ODBC SQL Server Driver]Function sequence error

Is this a multi-threaded issue? Can pandas.read_sql handle multiple simultaneous reads agains the same connection? maybe the question is whether it uses the same cursor or a different cursor for each read? The driver i am using is ceODBC

Answer 1

I was able to resolve this by opening/closing a connection each time i need to execute a query.

with ceODBC.connect(self.CONNECT_STR) as conn:
    result = pandas.read_sql(sql, conn)