Python MySQL INSERT from csv

Question

I am working on a script to parse a csv file and generate input for a MySQL table.

I import the data via csv.reader, so every row is a list of strings. I want to iterate over the rows and put different entries into the database.

I can get the following test to work:

sql = "INSERT INTO `testSmall` (`idtestSmall`, `column1`, `column2`) VALUES (1, 'entry1', 'entry2');"
cursor.execute (sql)

So my SQL connection works and the principle SQL syntax is ok. I can also access the entries I want to put in there, and they are correct and of the data type I expect. However, I don't seem to be able to use the same SQL syntax with variables within the iterations:

allData = csv.reader(open('TestTable.csv', 'rb'), delimiter=',', quotechar='|')
for row in allData:
    sql = "INSERT INTO `testSmall` (`idtestSmall`, `column1`, `column2`) VALUES (row[0], row[1], row[2]);"
    cursor.execute (sql)

This generates a Syntax Error:

Error 1064: You have an error in your SQL syntax; check the manual that corresponds to our MySQL server version for the right syntax to use near '[0], row[1], row[2])' at line 1

But the data types are correct and the SQL syntax is the same as in the working example...

Can anyone tell me what I'm doing wrong and how to make it work?

(In the end, I want to not only insert the pure csv entries but also derived values, which is why I'm not just using mysql bulk import.)

Thanks in advance for your help!

Answer 1

use:

sql = "INSERT INTO `testSmall` (`idtestSmall`, `column1`, `column2`) VALUES (?, ?, ?);"
cursor.execute (sql, (row[0], row[1], row[2]))

The questionmark is a placeholder. An extra advantage of using placeholders, is that they automatically make your input 'safe', by escaping qoutes etc.
Right now, you are using the row[0], row[1], row[2] as a string with the text "row[0], row[1], row[2]", instead of telling python to use the values of these variables.

---

Also, if you want to use rows of multiple lengths, or if you want to be able to easily change the size of your input list, you can dynamically create the placeholders:

sql = "INSERT INTO testSmall VALUES (%s);" % ', '.join('?' for _ in row)
cursor.execute (sql, row)

Answer 2

The way you are doing it, row[n] s don't refer to the variable row , but they are just a piece of string sent as it is to MySQL. (I bet you come from PHP background and expect the double quotes to replace your variables with their values).

You could do this to insert the values inside the string (any string):

sql = "INSERT INTO `testSmall` (`idtestSmall`, `column1`, `column2`) VALUES (%s, %s, %s);" % row # will map each %s to the `n`th element in `row`

(this will not work, be careful, because if row[0] is abc , that string will not be enclosed in quotes, so MySQL will not interpret it as a string). Try printing the sql variable, and copy/paste it into the mysql prompt to see if it will work.

However, when used with MySQL, you better escape these, like so:

sql = "INSERT INTO `testSmall` (`idtestSmall`, `column1`, `column2`) VALUES (%s, %s, %s);"
    cursor.execute(sql, row)

You can read more in the docs .