Python Importing a CSV file into a sqlite3 remove replicates

Question

I have a CSV file and I want to import this file into my sqlite3 database using Python. The column names of the CSV is the same with the column names of the database table, the following is the code i am using now.

df = pandas.read_csv(Data.csv)

df.to_sql(table_name, conn, index=False)

However it seems the command will import all data into the database, I am trying to only input the data that does not already exist in the database. Is there a way to do that without iterating every row of the csv or database?

Answer 1

Use the if_exists parameter.

df = pandas.read_csv(Data.csv)

df.to_sql(table_name,conn,if_exists='append',index=False)