dynamic csv to sqlite importing

Question

with open((path + 'stuff.csv'),'r+') as fin1:
    dr1 = csv.DictReader(fin1)
    create1="CREATE TABLE LGD ("
    exe1="INSERT INTO LGD ("
    for a in dr1.fieldnames:
        if a=='':
            pass
        elif ' ' in a:
            create1+=a.replace(" ","")+','
            exe1+=a.replace(" ","")+','
        else:
            create1 += a + ","
            exe1+= a + ","
    create1 = create1[:-1]
    exe1=exe1[:-1]
    create1 +=");"
        exe1+=") VALUES ("
    for a in range(len(dr1.fieldnames)-1):
        exe1+='?'+","
    exe1=exe1[:-1]
    exe1+=");"
    cur.execute(create1)
    to_db1 = [(a[dr1.fieldnames[0]],a[dr1.fieldnames[1]],a[dr1.fieldnames[2]],a[dr1.fieldnames[3]],a[dr1.fieldnames[4]]) for a in dr1]
    cur.executemany(exe1,to_db1)
    conn.commit()

ok my goal is to import each csv document as a table into sqlite database. Let's say I have 100 of them.

Right now my code writes the strings dynamically but how can i create the to_db1 list so I don't have to define how many columns there are?

specifically this line

to_db1 = [(a[dr1.fieldnames[0]],a[dr1.fieldnames[1]],a[dr1.fieldnames[2]],a[dr1.fieldnames[3]],a[dr1.fieldnames[4]]) for a in dr1]

I've seen this question, but mine is different because the fieldnames have spaces in them and they can't be takenout as many of the documents are uneditable. I also don't want to lose the list formatting shown below.

[(stuff,stuff,stuff),(stuff,stuff,stuff)]

Dynamically import columns from csv into database table in sqlite3 and python

to restate my question, I want to create the to_db1 list without defining the column names at compile time. it should create it in this format

[(stuff,stuff,stuff),(stuff,stuff,stuff)] 

at runtime.

Answer 1

To get the fieldnames without spaces and already joined by comma, you can use a list comprehension:

names = ['my _id', 'some a', 'some b']
sanitized_names = "({})".format(", ".join(x.replace(' ','') for x in names))

# => (my_id, somea, someb)

From there, you can reuse the answer you linked to.

To get tuples from the dicts in dr1 ordered like dr1.fieldnames , use [tuple(a[x] for x in dr1.fieldnames) for a in dr1] .