How do you cleanly pass column names into cursor, Python/SQLite?

Question

I'm new to cursors and I'm trying to practice by building a dynamic python sql insert statement using a sanitized method for sqlite3:

import sqlite3
conn = sqlite3.connect("db.sqlite")
cursor = conn.cursor()
list = ['column1', 'column2', 'column3', 'value1', 'value2', 'value3']
cursor.execute("""insert into table_name (?,?,?) 
values (?,?,?)""", list)

When I attempt to use this, I get a syntax error "sqlite3.OperationalError: near "?"" on the line with the values. This is despite the fact that when I hard code the columns (and remove the column names from the list), I have no problem. I could construct with %s but I know that the sanitized method is preferred.

How do I insert these cleanly? Or am I missing something obvious?

Answer 1

The (?, ?, ?) syntax works only for the tuple containing the values, imho... That would be the reason for sqlite3.OperationalError

I believe(!) that you are ought to build it similar to that:

cursor.execute("INSERT INTO {tn} ({f1}, {f2}) VALUES (?, ?)".format(tn='testable', f1='foo', f1='bar'), ('test', 'test2',))

But this does not solve the injection problem, if the user is allowed to provide tablename or fieldnames himself, however.

I do not know any inbuilt method to help against that. But you could use a function like that:

def clean(some_string):
    return ''.join(char for char in some_string if char.isalnum())

to sanitize the usergiven tablename or fieldnames. This should suffice, because table-/fieldnames usually consists only of alphanumeric chars.

Perhaps it may be smart to check, if

some_string == clean(some_string)

And if False, drop a nice exception to be on the safe side.

During my work with sql & python I felt, that you wont need to let the user name his tables and fieldnames himself, though. So it is/was rarely necessary for me.

If anyone could elaborate some more and give his insights, I would greatly appreciate it.

Answer 2

First, I would start by creating a mapping of columns and values:

data = {'column1': 'value1', 'column2': 'value2', 'column3': 'value3'}

And, then get the columns from here:

columns = data.keys()  
# ['column1', 'column3', 'column2']

Now, we need to create placeholders for both columns and for values:

placeholder_columns = ", ".join(data.keys())
# 'column1, column3, column2'

placeholder_values = ", ".join([":{0}".format(col) for col in columns])
# ':column1, :column3, :column2'

Then, we create the INSERT SQL statement:

sql = "INSERT INTO table_name ({placeholder_columns}) VALUES ({placeholder_values})".format(
    placeholder_columns=placeholder_columns,
    placeholder_values=placeholder_values
)

# 'INSERT INTO table_name (column1, column3, column2) VALUES (:column1, :column3, :column2)'

Now, what we have in sql is a valid SQL statement with named parameters. Now you can execute this SQL query with the data:

cursor.execute(sql, data)

And, since data has keys and values, it will use the named placeholders in the query to insert the values in correct columns.

Have a look at the documentation to see how named parameters are used. From what I can see that you only need to worry about the sanitization for the values being inserted. And, there are two ways to do that 1) either using question mark style, or 2) named parameter style.

Answer 3

So, here's what I ended up implementing, I thought it was pretty pythonic, but couldn't have answered it without Krysopath's insight:

    columns = ['column1', 'column2', 'column3']
    values = ['value1', 'value2', 'value3']
    columns = ', '.join(columns)
    insertString=("insert into table_name (%s) values (?,?,?,?)" %columns)
    cursor.execute(insertString, values)

Answer 4

    import sqlite3
    conn = sqlite3.connect("db.sqlite")
    cursor = conn.cursor()
## break your list into two, one for  column and one for value
    list = ['column1', 'column2', 'column3']
    list2= ['value1', 'value2', 'value3']
    cursor.execute("""insert into table_name("""+list[0]+""","""+list[1]+""","""+list[2]+""")
    values ("""+list2[0]+""","""+list2[1]+""","""+list2[2]+""")""")