Python SQLAlchemy: psycopg2.ProgrammingError relation already exists?

Question

I have repeatable tried to create a table MYTABLENAME with SQLAlchemy in Python. I deleted all tables through my SQL client Dbeaver but I am getting an error that the table exists such that

Traceback (most recent call last):
  File "/home/hhh/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/home/hhh/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: relation "ix_MYTABLENAME_index" already exists

sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "ix_MYTABLENAME_index" already exists
 [SQL: 'CREATE INDEX "ix_MYTABLENAME_index" ON "MYTABLENAME" (index)']

I succeed in the creation of tables and their insertions with an unique name but the second time I am getting the error despite deleting the tables in Dbeaver.

Small example

from datetime import date
from sqlalchemy import create_engine
import numpy as np
import pandas as pd

def storePandasDF2PSQL(myDF_):
    #Store results as Pandas Dataframe to PostgreSQL database.
    #
    #Example
    #df=pd.DataFrame(np.random.randn(8, 4), columns=['A','B','C','D'])
    #dbName= date.today().strftime("%Y%m%d")+"_TABLE"
    #engine = create_engine('postgresql://hhh:yourPassword@localhost:1234/hhh')
    #df.to_sql(dbName, engine)

    df      = myDF_
    dbName  = date.today().strftime("%Y%m%d")+"_TABLE"
    engine  = create_engine('postgresql://hhh:yourPassword@localhost:1234/hhh')
    # ERROR: NameError: name 'table' is not defined
    #table.declarative_base.metadata.drop_all(engine)    #Drop all tables

    #TODO: This step is causing errors because the SQLAlchemy thinks the 
    #TODO: table still exists even though deleted
    df.to_sql(dbName, engine)

What is the proper way to clean up the backend such as some hanging index in order to recreate the table with fresh data? In other words, how to solve the error?

Answer 1

The issue might be from sqlalchemy side which believes that there is an index as message of deletion of tables was not notified to the sqlalchemy. There is a sqlalchemy way of deleting the tables

table.declarative_base.metadata.drop_all(engine)

This should keep Sqlalchemy informed about the deletions.

Answer 2

This answer does not address the reusing of the same table names and hence not about cleaning up the SQLAlchemy metadata.

Instead of reusing the table names, add the execution time like this to the end of the tableName

import time 
dbName  = date.today().strftime("%Y%m%d")+"_TABLE_"+str(time.time())
dbTableName = dbName

so your SQL developmnet environment, such as SQL client locking up the connection or specific tables, does not matter that much. Closing Dbeaver can help while running the Python with SQLAlchemy.