How to insert a pandas dataframe into an existing postgres sql database?

Question

I have a dataframe like this

index  userID    OtherIDs
0   abcdef2035  [test650, test447, test968, test95]
1   abcdef3007  [test999, test992, test943, test834]
2   abcdef2006  [test175, test996, test986, test965]
3   abcdef2003  [test339, test968, test87, test678]
4   abcdef3000  [test129, test99, test921, test909]

The code that generates this dataframe will run each day. I need to upload this to a table name "result" in the existing database. I have to check if the table "result" exists, if it exists, delete/overwrite the values using the current values from the above dataframe.

creds of postgres db:

PGHOST = 'localhost'
PGDATABASE = 'TestDB'
PGUSER = 'postgres'
PGPASSWORD = 'admin1234'

Answer 1

you can use SQLAlchemy: ( https://docs.sqlalchemy.org/en/14/core/engines.html )

pandas df.to_sql: ( https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.to_sql.html )

assuming data frame name is df

from sqlalchemy import create_engine
engine = create_engine(user:password@host_ip:port/postgres_database)
df.to_sql('results', schema='<schema_name>', con = engine, if_exists='replace')

Just pass your credentials in the right format. ie engine = user:password@host_ip:port/postgres_database

to construct engine string: assuming following sign_in variables:

sign_in = {
  "database": "TestDB",
  "user": "postgres",
  "password": "<your_password>",
  "host": "localhost",
  "port": "<your_port>"
}

signin_info = 'postgresql+pygresql://'+sign_in['user']+':'+sign_in['password']+'@'+sign_in['host']+':'+sign_in['port']+'/'+sign_in['database']

from sqlalchemy import create_engine
engine = create_engine(signin_info)

df.to_sql('results', schema='<schema_name>', con = engine, if_exists='replace')