Using external variable in LOAD_DATA_INFILE

Question

I have this table structure:

date_sourced
sha1
vsdt
trendx
notes

And my csv structure: sha1,vsdt,trendx,notes

How can I insert a variable value to my date_sourced ? I tried this:

var  = "2018-1-10"
query = "LOAD DATA INFILE %s INTO TABLE jeremy_table_test FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (%s,sha1, @var1, trendx,notes) SET vsdt = TRIM(TRAILING ')' FROM TRIM(LEADING '(' FROM @var1))"
cursor.execute(query, (path,var))

but gives me error:

ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''2018-1-10',sha1, @var1, trendx,notes) SET vsdt = TRIM(TRAILING ')' FROM TRIM(LE' at line 1

Does LOAD_DATA_INFILE accept external variables? For example I have these two variables

import csv
import mysql.connector

path = 'C:\\Users\\trendMICRO\\Desktop\\OJT\\updated_test.csv'

print "CSV importing to database"


mydb = mysql.connector.connect(user='root', password='',
                            host='localhost',
                            database='jeremy_db')
cursor = mydb.cursor()
var  = "apple"
query = "LOAD DATA INFILE %s INTO TABLE jeremy_table_test FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (%s, @var1, person) SET vsdt = TRIM(TRAILING ')' FROM TRIM(LEADING '(' FROM @var1))"
cursor.execute(query, (path))
mydb.commit()

How can I apply it here in my query,by replacing 'path/to/rb' to my variable path and value of fruit set by my variable var = "apple" ?

LOAD DATA INFILE 'path/to/rb' INTO TABLE jeremy_table_test FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (fruit, @var1, person) SET vsdt = TRIM(TRAILING ')' FROM TRIM(LEADING '(' FROM @var1))
cursor.execute(query)
connection.commit()

Answer 1

This answer assumes you are using the MySQLdb module, if you are using a different driver the answer may vary.

To add the values we want to use a parameterized query as follows:

var  = "apple"
path = "C:\\apple.txt"
query = "LOAD DATA INFILE %s INTO TABLE jeremy_table_test FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (%s, @var1, person) SET vsdt = TRIM(TRAILING ')' FROM TRIM(LEADING '(' FROM @var1))"
cursor.execute(query, (path, var))
connection.commit()

The parameters stored in the tuple given as the second argument to cursor.execute() will be substituted for the values of %s that occur in the query string.

Unfortunately this won't work because path is not going to be used as a column value, and according to the MySQLdb user's guide :

Parameter placeholders can only be used to insert column values. They can not be used for other parts of SQL, such as table names, statements, etc.

So we need to do a horrible thing and write our query string by hand using the file name. This is unsafe if you are allowing user input to be passed in to a variable like path .

We can still use a parameter for the value of var as before.

var  = "apple"
path = "C:\\apple.txt"
query = "LOAD DATA INFILE '" + path + "' "
query += "INTO TABLE jeremy_table_test FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (%s, @var1, person) SET vsdt = TRIM(TRAILING ')' FROM TRIM(LEADING '(' FROM @var1))"
cursor.execute(query, (var,))
connection.commit()