Pyodbc execution failed on temp table

Question

I have a Python Script which uses the pyodbc library to run some queries which I have task scheduler running on a set schedule. The script had been working fine without issue all of last week and it suddenly encountered the error:

DatabaseError: Execution failed on sql 'select * from #output  ('42S02', "[42S02] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Invalid object name '#output'. (208) (SQLExecDirectW)")

This error arises when I try to extract that data from the query using pd.read_sql . The very odd part is that no modifications were made to the script at all. It had been working just fine up until today.

I researched similar problems on stackoverflow and attempted using the same solutions (creating a global temp table instead of a local one) and the issue persists. Strangely, the query includes two temp tables, #data and #output and I'm able to access to contents of #data using pd.read_sql with no problem, but for whatever reason, attempting to use the same method on the other temp table results in the error above.

The code itself might be a bit long to include on here, but this is what it resembles for the most part (I'll be happy to share the entire thing if it's not an issue):

connection = pyodbc.connect(...)
cursor = connection.cursor()

query = """create table #data (...)

         ... #some other commands here in the middle

         create table #output (...)

         ... #two more lines of SQL commands and that's it
         """

cursor.execute(query)
totals = pd.read_sql("""select * from #output"""), connection)

The line cursor.execute(query) doesn't appear to generate an error as I get <pyodbc.Cursor at 0xef2eea0> if I run up to that line only.

I appreciate the assistance.

EDIT: I suspect the error message is due to the fact that there is a while loop portion before the create table #output statement and it is somehow preventing the next commands from executing properly. I've included a bit more detail using my example code from above:

connection = pyodbc.connect(...)
cursor = connection.cursor()

query = """create table #data (...)

           ... #some other commands here in the middle

           declare @order int, @limit int, @check varchar(20)
           set @order = 1
           set @limit = (select count(*) from #data where [weekday] = 'Friday')

           while @order <= @limit
           begin

           update #data
           set [order] = @order
           where #data.[date] = (select top 1 [date] from #data where [order] is null order by [date] asc)

           if (select top 1 [weekday] from #data where [order] is not null order by [date] desc) = 'Friday'
                 set @order = @order + 1

           update #data
           set weekly_desc = (select top 1 weekly_desc from #data where [weekday] = 'Friday' and [order] is null order by [date] asc)
           where [order] = @order

           end

           create table #output (...)
              """

cursor.execute(query)
totals = pd.read_sql("""select * from #output""", connection)

Answer 1

After some more digging, I found that after updating the #data table (the step right before #create table #output... , if I had Python execute pd.read_sql("""select * from #data""", connection) , the console showed that the final few rows of #data contained NaN values in the order column. I have no idea why it would since running the query in SQL Server executes without issue. This probably explains why it couldn't proceed to the next step which was create table #output .

My only way to get it to work again was to rewrite part of the query, specifically the while loop portion, by changing it from

while @order <= @limit
begin

update #data
set [order] = @order
where #data.[date] = (select top 1 [date] from #data where [order] is null order by [date] asc)

if (select top 1 [weekday] from #data where [order] is not null order by [date] desc) = 'Friday'
set @order = @order + 1

update #data
set weekly_desc = (select top 1 weekly_desc from #data where [weekday] = 'Friday' and [order] is null order by [date] asc)
where [order] = @order

end

to

while @order <= @limit
begin

update #data
set [order] = @order
where #data.[date] = (select top 1 [date] from #data where [order] is null order by [date] asc)

if (select top 1 [weekday] from #data where [order] is not null order by [date] desc) = 'Friday'
set @order = @order + 1

end

update #data
set weekly_desc = t.weekly_desc
from (select [order], weekly_desc 
      from #data
      where weekly_desc is not null
      ) as t
where #data.weekly_desc is null
and #data.[order] = t.[order]

The create table #output step and other commands came after and I was able to have the entire script run without issue. Hoping it doesn't mysteriously stop working again.

Thank you all for your help!