Android - OutOfMemory when reading text file

Question

I'm making a dictionary app on android. During its startup, the app will load content of .index file (~2MB, 100.000+ lines)

However, when i use BufferedReader.readLine() and do something with the returned string, the app will cause OutOfMemory.

// Read file snippet
Set<String> indexes = new HashSet<String)();

FileInputStream is = new FileInputStream(indexPath);
BufferedReader reader = new BufferedReader(new InputStreamReader(is));

String readLine;

while ( (readLine = reader.readLine()) != null) {
    indexes.add(extractHeadWord(readLine));
}

// And the extractHeadWord method
private String extractHeadWord(String string) {
    String[] splitted = string.split("\\t");
    return splitted[0];
}

When reading log, I found that while executing, it causes the GC explicitly clean objects many times (GC_EXPLICIT freed xxx objects, in which xxx is a big number such as 15000, 20000).

And I tried another way:

final int BUFFER = 50;
char[] readChar = new char[BUFFER];

//.. construct BufferedReader

while (reader.read(readChar) != -1) {
    indexes.add(new String(readChar));
    readChar = new char[BUFFER];
}

..and it run very fast. But it was not exactly what I wanted.

Is there any solution that run fast as the second snippet and easy to use as the first?

Regard.

Answer 1

The extractHeadWord uses String.split method. This method does not create new strings but relies on the underlying string (in your case the line object) and uses indexes to point out the "new" string.

Since you are not interessed in the rest of the string you need to discard the it so it gets garbage collected otherwise the whole string will be in memory (but you are only using a part of it).

Calling the constructor String(String) ("copy constructor") discards the rest of string:

private String extractHeadWord(String string) {
    String[] splitted = string.split("\\t");
    return new String(splitted[0]);
}

Answer 2

What happens if your extractHeadWord does this return new String(splitted[0]); .

It will not reduce temporary objects, but it might reduce the footprint of the application. I don't know if split does about the same as substring, but I guess that it does. substring creates a new view over the original data, which means that the full character array will be kept in memory. Explicitly invoking new String(string) will truncate the data.