What is most efficient way for loading big amount of data from CSV file in Android?

Question

I have 15MB csv file (350 000 lines). I have problem with loading this data. I have tried:

1) Loading file data to memory: quite fast (about 7 seconds), but I'm receiving OutOfMemory error and OS closes my app. Even without this error, it's not perfect solution to load data for 7 seconds during each application start.

2) Loading to SQLite DB: would be perfect(only one load is required), but loading process takes more than 1 minute ( I'm using db.begintransaction/endtransaction to speed up process).

3) I can't ship app with preloaded data, because this CSV contains feeds, which should be updated periodically.

I was testing app with Samsung S3. Not the slowest smartphone.

------ Update

Possibly there are special tools for bulk load of CSV data to database??

Answer 1

I can't ship app with preloaded data, because this CSV contains feeds, which should be updated periodically.

As I understand you need to do this background operation periodically using service and sync data with server and at the same time insert/update received records to SQLite database.

First time when database is empty then you can give progress bar with appropriate message that data is being retrieved and copied into local database. Once this done then display data which has been received.

Later, on you can use SyncAdapter to keep sync your application database with server or run a service periodically.