如何使用 python 将 csv 数据导入 postgres 数据库

Question

I am trying to import the data in my excel sheet to the postgresql database using python, and when i do that i get the following error.

i have already converted my excel to csv and then tried using the 'copy' statement to import the data to postgres database.

import psycopg2

conn = psycopg2.connect("host=localhost dbname=djangotest user=postgres password=*******")
cur = conn.cursor()

with open('C:\\Users\\********\\Desktop\\excelsheet.csv', 'r') as f:
    next(f) # Skip the header row.
    cur.copy_from(f, 'us_arrays', sep=',')

conn.commit()

psycopg2.errors.BadCopyFileFormat: missing data for column "ip_address_or_service_machine"
CONTEXT:  COPY us_arrays, line 1: "(CMDB)",.Device Type,.Frame or Data Tier,.Corp Device,.Encrypt Enabled,.Dedicated Device,".IP Addres..."```

Answer 1

Judging by the error text ( missing data for column ), it seems like the number of columns in us_arrays table doesn't match the number of columns in your CSV file. You can also use columns keyword attribute when calling copy_from to specify the database table columns which should be populated from your file. Read more about it here .

A common scenario would be having a database table with 3 columns, eg id , num , data and a CSV file with only 2 columns num and data . Without specifying columns , the copy_from function would import num from the CSV file into id database column, data into num , and there would be no data left to import into data database column.

Answer 2

Your question is tagged as Django, so I'll consider you want to import something linked to Model .

I give a detail explanation on how to speed up XL loading in this response Now, if performance is not an issue (your dataset is not too big, and do not have FKs), you can simply use Django Import Export

Adding mixins to your Admin, you won't have to bother about file conversion and will give you diff when loading your file