How to read a large json from a text file in android app?

Question

I have a text file in my android app which consist a json. I need to read and parse that json. File size is 21 mb. I am using following code to read file:

 StringBuilder stringBuilder = new StringBuilder();
 InputStream input = getAssets().open(filename);
 int size = input.available();
 byte[] buffer = new byte[size];
 byte[] tempBuffer = new byte[1024];
 int tempBufferIndex = 0;
 for(int i=0; i<size; i++){
    if(i == 0){
        tempBuffer[tempBufferIndex] = buffer[i];
    }else{
        int mod = 1024 % i;
        if(mod == 0){
            input.read(tempBuffer);
            stringBuilder.append(new String(tempBuffer));
            tempBufferIndex = 0;
        }
        tempBuffer[tempBufferIndex] = buffer[i];
    }
}
input.close();

Size int is 20949874 in real case. After loop is done stringBuilder length is always 11264 even if i change range of for loop. I tried to make one String from InputStream without using loop but it always gives me OutOfMemoryError Exception. I also get "Grow heap (frag case) to 26.668MB for 20949890-byte allocation" in my logs. I searched here and tried different solutions but did not make it work. Any idea how should i solve this issue. Thanks in advance.

Answer 1

For big json files you should use SAX parser and not DOM. For example JsonReader .

DOM (“Document Object Model”) loads the entire content into memory and permits the developer to query the data as they wish. SAX presents the data as a stream: the developer waits for their desired pieces of data to appear and saves only the parts they need. DOM is considered easier to use but SAX uses much less memory.

Answer 2

You can try to split the file into several parts. So during processing the app hopefully doesn't get out of memory.

You should also consider using "largeHeap" flag in your manifest (See http://developer.android.com/guide/topics/manifest/application-element.html )

I don't know your file, but maybe if you use smaller JSON tags, you can reduce storage as well.