python sqlite: write a big list to sqlite database

Question

i am new to sqlite and i think this question should have been answered before but i havent been able to find an answer. i have a list of around 50 elements that i need to write to an sqlite database with 50 columns. went over the documentation @ https://docs.python.org/2/library/sqlite3.html but in the examples the values are specified by ? (so for writing 3 values, 3 ? are specified

sample code:

row_to_write = range(50) conn = sqlite3.connect('C:\\sample_database\\sample_database') c = conn.cursor()

tried these:

approach 1

c.execute("INSERT INTO PMU VALUES (?)", row_to_write) ERROR: OperationalError: table PMU has 50 columns but 1 values were supplied

approach 2...tried writing a generator for iterating over list

def write_row_value_generator(row_to_write): for val in row_to_write: yield (val,) c.executemany("INSERT INTO PMU VALUES (?)", write_row_value_generator(row_to_write)) ERROR: OperationalError: table PMU has 50 columns but 1 values were supplied What is the correct way of doing this?

Answer 1

You need to specify the names of the columns. Sqlite will not guess those for you.

columns = ['A', 'B', 'C', ...]
n = len(row_to_write)
sql = "INSERT INTO PMU {} VALUES ({})".format(
    ', '.join(columns[:n]) , ', '.join(['?']*n))

c.execute(sql, row_to_write)

Note also that if your rows have a variable number of columns, then you might want to rethink your database schema. Usually each row should have a fixed number of columns, and the variability expresses itself in the number of rows inserted, not the number of columns used.

For example, instead of having 50 columns, perhaps you need just one extra column, whose value is one of 50 names (what used to be a column name). Each value in row_to_write would have its own row, and for each row you would have two columns: the value and the name of the column.

Answer 2

Assuming that your row_to_write has exactly the same number of items as PMU has columns, you can create a string of ? marks easily using str.join : ','.join(['?']*len(row_to_write))

import sqlite3

conn = sqlite3.connect(':memory:')
c = conn.cursor()
c.execute("create table PMU (%s)" % ','.join("col%d"%i for i in range(50)))

row_to_write = list(range(100,150,1))
row_value_markers = ','.join(['?']*len(row_to_write))
c.execute("INSERT INTO PMU VALUES (%s)"%row_value_markers, row_to_write)

conn.commit()