How to insert large amount of data in sqlite database in Android

Question

I have a lot of data that is stored in a CSV file (about 20,100 rows), which I need to insert into a sqlite database.

This insert is taking very long to complete. What is the fastest way to insert this data?

Answer 1

As you have suggested, number of rows are huge I will recommend not to use AsyncTask , as its not tied to your Activity lifecycle ie if you activity which started it dies, it doesnt mean AsyncTask dies as well, so if you try initiate a AsyncTask and somehow if your activity dies eg screen rotation or back key pressed, upon restarting another AsyncTask will get spawned rather then it getting linked to already executing AsyncTask . hence duplicating same operations.

So, all in all I would recommend following approach

(A)

1. Create a IntentService , it's handleIntent() api already executes in a worker thread so you don't have to worry about any thing, and once all messaged in its queue are finished it automatically dies, so no worry at all about leaking any resources.

2. write your logic for inserting rows in bulk, use content resolver bulkInsert() api for same. I will recommend inserting in 100 roes per batch, you can implement rollback and error checks to make sure insert goes normally.

3. Once all insert is finish, you can post back to your UI using Handler and Messengers.

with all this you will achieve two major challenge

1. Not to hang up your UI, escaping any possible ANR
2. Even if back key is pressed, ensured that db operation goes on smoothly as it was taken up in background task.

Answer 2

Using AsyncTask<> , insert 20,100 rows inserts in database. Using this asynctask whole work run in background. For more information follow this link

Answer 3

The best solution would be using services and executor because as OP described, process can take a lot time. Thanks that You will be able to close app or move it to background with no worried Your long process is destroyed.

Using AsyncTask is not a good idea because it was designed for short operations as it is described on http://developer.android.com/reference/android/os/AsyncTask.html You must also be careful with using it. Changing orientation screen cause recreating view and also task of asynctask.

AsyncTasks should ideally be used for short operations (a few seconds at the most.) If you need to keep threads running for long periods of time, it is highly recommended you use the various APIs provided by the java.util.concurrent package such as Executor, ThreadPoolExecutor and FutureTask.