Why is PhoneGap Android app crashing while inserting bunch of data into SQL?

Question

I have PhoneGap app running in Android. When the app starts, it inserts approximately 500 rows into SQL. Table has 10 columns. It's not a lot of data, I have a JSON in textfile and its size is about 120 kB. I thought this couldn't touch any limits at all, but there is probably some limitation I don't know about or maybe a bug in Android, because this same app runs on some versions of Android (2.2) without problem, but crashes immediately or in few minutes when working with the SQL database on other versions of Android (1.6, 2.1, some 2.3 probably more...)

Here is the code I use to populate the DB that crashes on Android 1.6:

db = window.openDatabase("db", "1.0", "Description", 1000000); 
$.get('db/app_data.dat',function(result){
  var data = $.parseJSON(result);
  try {
    db.transaction(function(tx){
      $.each(data.items, function(i, v){
        try {
          tx.executeSql('INSERT INTO table(c1,c2,c3, ...) VALUES (?,?,?, ...)',[v.c1, v.c2, v.c3, ...]);
        } catch(e) {
          alert(e.message);
        }
      });
    });
  } catch(e) {
    alert(e.message);
    return;
  }
});

Can anybody help me? Is there some limit I don't know about? Or am I doing something wrong when inserting data in the SQL database?

EDIT:

Here is output of LogCat, I think these are some important lines from log when app crashes. However, I have no deeper knowledge of Java and Android:

WARN/dalvikvm(1525): ReferenceTable overflow (max=512)
WARN/dalvikvm(1525): Last 10 entries in JNI local reference table:
WARN/dalvikvm(1525):   502: 0x4375cbb8 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   503: 0x4374c9a0 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   504: 0x4377c5c0 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   505: 0x437c3040 cls=Ljava/lang/String; (36 bytes)
WARN/dalvikvm(1525):   506: 0x43760bd8 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   507: 0x437625e8 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   508: 0x43762608 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   509: 0x43762628 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   510: 0x43759178 cls=Ljava/lang/String; (28 bytes)
WARN/dalvikvm(1525):   511: 0x43766808 cls=Landroid/webkit/WebViewCore; (116 bytes)
ERROR/dalvikvm(1525): Failed adding to JNI local ref table (has 512 entries)
INFO/dalvikvm(1525): "WebViewCoreThread" prio=5 tid=15 RUNNABLE
INFO/dalvikvm(1525):   | group="main" sCount=0 dsCount=0 s=N obj=0x437668a0 self=0x1b0bd0
INFO/dalvikvm(1525):   | sysTid=1532 nice=0 sched=0/0 handle=1772784
INFO/dalvikvm(1525):   at android.webkit.LoadListener.nativeFinished(Native Method)
INFO/dalvikvm(1525):   at android.webkit.LoadListener.tearDown(LoadListener.java:1076)
INFO/dalvikvm(1525):   at android.webkit.LoadListener.handleEndData(LoadListener.java:642)
INFO/dalvikvm(1525):   at android.webkit.LoadListener.handleMessage(LoadListener.java:203)
INFO/dalvikvm(1525):   at android.os.Handler.dispatchMessage(Handler.java:99)
INFO/dalvikvm(1525):   at android.os.Looper.loop(Looper.java:123)
INFO/dalvikvm(1525):   at android.webkit.WebViewCore$WebCoreThread.run(WebViewCore.java:471)
INFO/dalvikvm(1525):   at java.lang.Thread.run(Thread.java:1060)
ERROR/dalvikvm(1525): VM aborting
DEBUG/Zygote(30): Process 1525 terminated by signal (11)

Answer 1

Phobos' answer has some pertinent information, but it doesn't include how it applies to your problem or how to fix your specific problem. I ran into a similar issue while doing some performance testing in android javascript that calls across to java via an object bound into the webview context with addJavascriptInterface. Although I am not explicitly using JNI, apparently under the covers the bound interface uses JNI to marshall the data across - I was getting an error and stack trace similar to yours. Apparently, webDB or localStorage or whatever you are doing there also uses JNI under the covers.

As per the info that Phobos linked, there is a finite limit to the number of references you can have in scope when you are using JNI. In the case of javascript that indirectly uses JNI, it seems that the references must be kept around based on the javascript scope. Looking at your code, I'm not sure exactly what you are doing to exceed the limit, but it looks like it could be either the number of calls you make in the transaction overall or your values arrays end up having more than 512 elements. If you need to preserve transactional integrity and you really have over 500 operations you might be out of luck. However I suspect that you should be able to find a way to avoid the 512 limit by rewriting your code to build up a query string outside of the scope of the tx.executeSql call.

Maybe you could rewrite the code to build your query strings and then actually call tx.executeSql inside an anonymous function wrapper? You probably need to build strings containing the values instead of using the 'INSERT INTO table() VALUES()',[] syntax... As I said above, if you have more than 500 operations in a transaction, you might still run into trouble, but it would be worth a try!

PS For reference, here is a version of my code that was crashing along with the fix:

var test = function() {
    var a = "a";
    for(i=0;i<1000;i++) {
        boundJavaObj.test(a);
    }
}

test() crashes with an error like:

03-28 10:57:45.634: W/dalvikvm(21294): ReferenceTable overflow (max=512)

var test2 = function() {
    var a = "a";
    for(i=0;i<1000;i++) {
        (function(){boundJavaObj.test(a);})();
    }
}

test2() doesn't crash.

Answer 2

Maybe you need a recursive callback function like this:

//too many insert statements
var sqlits= ["INSERT INTO ...", .. .. .. ,"INSERT INTO ..."];

function populate(){
    db.transaction(function(tx){
        if(sqlits.length > 0){
            sql=sqlits.shift();
            tx.executeSql(sql);
        }else{
            alert('End');
        }
        return true;
    },function(tx,err){ 
        console.log("SQL Error: "+err); 
    },populate);
};

populate();

Answer 3

OK. Found the answer. It's a limit of 512 local references. What you need to do is convert them to global references using NewGlobalRef(), or to clean up the locals shortly after creating them if they're not needed. Changing them to global references will leak memory however.

Check out this thread for more information on this known issue.