SQLite获取ID并在使用executemany时插入

Question

I am optimising my code, and reducing the amount of queries. These used to be in a loop but I am trying to restructure my code to be done like this. How do I get the second query working so that it uses the id entered in the first query from each row. Assume that the datasets are in the right order too.

self.c.executemany("INSERT INTO nodes (node_value, node_group) values (?, (SELECT node_group FROM nodes WHERE node_id = ?)+1)", new_values)
#my problem is here
new_id = self.c.lastrowid
connection_values.append((node_id, new_id))
#insert entry
self.c.executemany("INSERT INTO connections (parent, child, strength) VALUES (?,?,1)", connection_values)

These queries used to be a for loop but were taking too long so I am trying to avoid using a for loop and doing the query individually. I believe their might be a way with combining it into one query but I am unsure how this would be done.

Answer 1

You will need to either insert rows one at a time or read back the rowids that were picked by SQLite's ID assignment logic; as documented in Autoincrement in SQLite , there is no guarantee that the IDs generated will be consecutive and trying to guess them in client code is a bad idea.

You can do this implicitly if your program is single-threaded as follows:

Set the AUTOINCREMENT keyword in your table definition. This will guarantee that any generated row IDs will be higher than any that appear in the table currently.

1. Immediately before the first statement, determine the highest ROWID in use in the table.

   oldmax ← Execute( "SELECT max(ROWID) from nodes" ).

2. Perform the first insert as before.

3. Read back the row IDs that were actually assigned with a select statement:

   NewNodes ← Execute( "SELECT ROWID FROM nodes WHERE ROWID > ? ORDER BY ROWID ASC" , oldmax ) .

4. Construct the connection_values array by combining the parent ID from new_values and the child ID from NewNodes .

5. Perform the second insert as before.

This may or may not be faster than your original code; AUTOINCREMENT can slow down performance, and without actually doing the experiment there's no way to tell.

If your program is writing to nodes from multiple threads, you'll need to guard this algorithm with a mutex as it will not work at all with multiple concurrent writers.