Cannot drop table in pandas to_sql using SQLAlchemy

Question

I'm trying to drop an existing table, do a query and then recreate the table using the pandas to_sql function. This query works in pgadmin, but not here. Any ideas of if this is a pandas bug or if my code is wrong?

Specific error is ValueError: Table 'a' already exists.

import pandas.io.sql as psql
from sqlalchemy import create_engine

engine = create_engine(r'postgresql://user@localhost:port/dbname')

c = engine.connect()
conn = c.connection

sql = """
drop table a;
select * from some_table limit 1;
"""
df = psql.read_sql(sql, con=conn)
print df.head()
df.to_sql('a', engine)

conn.close()

Answer 1

Why are you doing this like that? There is a shorter way: the if_exists kwag in to_sql . Try this:

import pandas.io.sql as psql
from sqlalchemy import create_engine

engine = create_engine(r'postgresql://user@localhost:port/dbname')

c = engine.connect()
conn = c.connection

sql = """
select * from some_table limit 1;
"""
df = psql.read_sql(sql, con=conn)
print df.head()
# Notice how below line is different. You forgot the schema argument
df.to_sql('a', con=conn, schema=schema_name, if_exists='replace')

conn.close()

According to docs :

replace: If table exists, drop it, recreate it, and insert data.

---

Ps. Additional tip:

This is better way to handle the connection:

with engine.connect() as conn, conn.begin():
    sql = """select * from some_table limit 1"""
    df = psql.read_sql(sql, con=conn)
    print df.head()
    df.to_sql('a', con=conn, schema=schema_name, if_exists='replace')

Because it ensures that your connection is always closed, even if your program exits with an error. This is important to prevent data corruption. Further, I would just use this:

import pandas as pd
...
pd.read_sql(sql, conn)

instead of the way you are doing it.

So, if I was in your place writing that code, it would look like this:

import pandas as pd
from sqlalchemy import create_engine

engine = create_engine(r'postgresql://user@localhost:port/dbname')

with engine.connect() as conn, conn.begin():
    df = pd.read_sql('select * from some_table limit 1', con=conn)
    print df.head()
    df.to_sql('a', con=conn, schema=schema_name, if_exists='replace')