在Python中将CSV记录导入MySQL数据库

Question

我正在尝试读取CSV文件并将记录导入mysql表。 CSV文件看起来像，

db_columns.csv： -

ID,SCHOOL_CODE,DISTNAME,AC_YEAR,SCHOOL_NAME,STATE_NAME,BLOCK_NAME,CLUSTER_NAME,VILLAGE_NAME,PINCODE,RURURB,ELECTRIC_YN,SCHMGT,LOWCLASS,HIGHCLASS,COMPUTER,CAL_YN,MEDINSTR1,MEDINSTR2,MEDINSTR3,MEDINSTR4
0,35010103903,ANDAMANS,2015-16,PRIMARY SCHOOL INVENT PUBLIC (PU),Andaman And Nicobar,BRC- SOUTH ANDAMAN,"DIET, GARACHARMA",BATHUBASTHI,744105,2,1,5,1,5,4,1,19,98,98,98
1,35010101902,ANDAMANS,2015-16,GOVT PRIMARY SCHOOL MAKKA PAHAR,Andaman And Nicobar,BRC- SOUTH ANDAMAN,GSSS RANGACHANG,CALICUT,744105,1,1,1,1,5,0,2,16,19,98,98

MySQL表看起来像，

+--------------+--------------+
| column_name  | column_type  |
+--------------+--------------+
| ID           | double       |
| SCHOOL_CODE  | double       |
| DISTNAME     | varchar(100) |
| AC_YEAR      | varchar(25)  |
| SCHOOL_NAME  | varchar(300) |
| STATE_NAME   | varchar(100) |
| BLOCK_NAME   | varchar(200) |
| CLUSTER_NAME | varchar(200) |
| VILLAGE_NAME | varchar(200) |
| PINCODE      | varchar(100) |
| RURURB       | double       |
| ELECTRIC_YN  | double       |
| SCHMGT       | double       |
| LOWCLASS     | double       |
| HIGHCLASS    | double       |
| COMPUTER     | double       |
| CAL_YN       | double       |
| MEDINSTR1    | double       |
| MEDINSTR2    | double       |
| MEDINSTR3    | double       |
| MEDINSTR4    | double       |
+--------------+--------------+
21 rows in set (1.20 sec)

我写的Python代码，

#!/usr/bin/python
import MySQLdb
import csv

db = MySQLdb.connect(host="localhost",    # your host, usually localhost
                     user="admin",         # your username
                     passwd="admin",  # your password
                     db="School_Test")        # name of the data base

cur = db.cursor()

csv_data = csv.reader(file('db_columns.csv'))

for row in csv_data:

    cur.execute ("INSERT INTO part_table_test (ID,SCHOOL_CODE,DISTNAME,AC_YEAR,SCHOOL_NAME,STATE_NAME,BLOCK_NAME,CLUSTER_NAME,VILLAGE_NAME,PINCODE,RURURB,ELECTRIC_YN,SCHMGT,LOWCLASS,HIGHCLASS,COMPUTER,CAL_YN,MEDINSTR1,MEDINSTR2,MEDINSTR3,MEDINSTR4) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)",row)

db.close()

数据库和表名称如下，

DB Name :- School_Test
Table Name :- part_table_test
CSV File Name :- db_columns.csv

当我运行代码时，它给了我错误： -

_mysql_exceptions.DataError: (1265, "Data truncated for column 'ID' at row 1")

我试图解决这个错误。 应该建议的方法是什么？

Answer 1

当您创建阅读器时，它将遍历您的csv文件的所有行，包括第一行，其中包含标题。 尝试通过在for循环之前调用next来丢弃它们：

csv_data = csv.reader(file('db_columns.csv'))
next(csv_data) # discard header
for row in csv_data:
    cur.execute ("INSERT INTO part_table_test (ID,SCHOOL_CODE,DISTNAME,AC_YEAR,SCHOOL_NAME,STATE_NAME,BLOCK_NAME,CLUSTER_NAME,VILLAGE_NAME,PINCODE,RURURB,ELECTRIC_YN,SCHMGT,LOWCLASS,HIGHCLASS,COMPUTER,CAL_YN,MEDINSTR1,MEDINSTR2,MEDINSTR3,MEDINSTR4) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)",row)

Answer 2

这通常意味着CSV中的ID字段的字符数多于该列的数据库中允许的最大字符数。

1）尝试改变

    csv_data = csv.reader(file('db_columns.csv'), delimiter=',')

2）同时将ID的类型更改为INT或BIGINT

3）不要忘记跳过初始行（标题），因为它会将列名插入数据库并失败

    count = 0
    for row in csv_data:
        if count < 1:
            continue
        else:
            cur.execute ("INSERT INTO part_table_test (ID,SCHOOL_CODE,DISTNAME,AC_YEAR,SCHOOL_NAME,STATE_NAME,BLOCK_NAME,CLUSTER_NAME,VILLAGE_NAME,PINCODE,RURURB,ELECTRIC_YN,SCHMGT,LOWCLASS,HIGHCLASS,COMPUTER,CAL_YN,MEDINSTR1,MEDINSTR2,MEDINSTR3,MEDINSTR4) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)",row)
        count+=1

Answer 3

跳过CSV文件中的第一行，该标题行，然后处理剩余的行。

  csv_data = csv.reader(file('db_columns.csv'))

  # retrieve the header line and discard it
  next(csv_data, None)

  for row in csv_data:

---

我认为错误来自第一行，字符串值"ID"被分配给double。 这将在MySQL中评估为0，但该转换将导致MySQL抛出错误或警告。 行为取决于会话的sql_mode设置。