OutOfMemoryError while JSON parsing in android

Question

I am using the below code to parse the JSON String fetched from Web, (30,000 records)

DefaultHttpClient   httpclient = new DefaultHttpClient(new BasicHttpParams());
        HttpPost httppost = new HttpPost(params[0]);
        httppost.setHeader("Content-type", "application/json");
        InputStream inputStream = null;
        String result = null;
        HttpResponse response = null;
        try {
            response = httpclient.execute(httppost);

        } catch (ClientProtocolException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }           
        HttpEntity entity = response.getEntity();
        try {
            inputStream = entity.getContent();
        } catch (IllegalStateException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
        BufferedReader reader = null;
        try {
            reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"),8);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }

        StringBuilder sb = new StringBuilder();
        String line = null;
        try {
            while ((line = reader.readLine()) != null)  
            {
                sb.append(line + "\n");
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        result = sb.toString();

I am getting the OutofMemory error in the below code

while ((line = reader.readLine()) != null)  
{
    sb.append(line + "\n");
}

How to get rid of this error.This error does occur when the json string is very huge as it contains data of about 30,000 records.

Any help in this regard is highly appreciated..

Answer 1

Android imposes a memory cap limit (of 16 MB in almost all phones, with some newer tablets have more) for each and every application. Application should make sure they maintain their live memory limit below that level.

So we can't hold a big string, say over 1MB, in full sometimes since the total live memory usage of applicaiton may exceed that limit. Remember, the total memory usage includes all objects (including UI elements) we allocated in our app.

So your only solution is to use a Streaming JSON parser, which takes data as it comes. That is you should not hold on full string in a String object. One option is to use Jackson JSON parser .

EDIT : Android now support JSONReader from API level 11. Never used it, but it seems the way to go..

Answer 2

If data file is too large, you cannot read it all to memory.

Read a line and then write it to a native file. Do not use a StringBuilder to hold all data in memory.

Answer 3

Try to import your data in chuncks, like 1000 records each time. Hopefully you will not experince this issue.

Answer 4

I solved this problem with this library . There is a very good tutorial here .

With this you will bypass converting entity.getContent() to String and that will solve your problem.

InputStream inputStream = entity.getContent();
JsonReader reader = Json.createReader(inputStream);
JsonObject jsonObject = reader.readObject();
return jsonObject;