Python SQLite3 - Does it BEGIN, END transaction on each insert statement?

Question

I am trying to optimize my SQLite3 inserts as I am working on some large data set.

What I am wondering is if the .execute("INSERT...") statements BEGIN and END a transaction for each execute statement? Or does it simply execute a BEGIN on the first execute, and it waits for the .commit() operation, hence I can do 10,000 .execute commands and then do .commit(), hence I only did one transaction?

I am not very clear on that side of the story and I would really appreciate any input on this.

Answer 1

The documentation says:

By default, the sqlite3 module opens transactions implicitly before a Data Modification Language (DML) statement (ie INSERT / UPDATE / DELETE / REPLACE ), and commits transactions implicitly before a non-DML, non-query statement (ie anything other than SELECT or the aforementioned).

Answer 2

From SQLite Optimization FAQ[1]:

Unless already in a transaction, each SQL statement has a new transaction started for it. This is very expensive, since it requires reopening, writing to, and closing the journal file for each statement. This can be avoided by wrapping sequences of SQL statements with BEGIN TRANSACTION; and END TRANSACTION; statements. This speedup is also obtained for statements which don't alter the database.

So, there are two ways:

1. Use executemany() method.
2. Use explicit BEGIN and COMMIT statement.

[1] http://web.utk.edu/~jplyon/sqlite/SQLite_optimization_FAQ.html