Does Immutability of Strings in Java cause Out Of Memory

Question

I have written a simple Java program that reads a million rows from the Database and writes them to a File.

The max memory that this program can use is 512M.

I frequently notice that this program runs Out Of Memory for more than 500K rows.

Since the program is a very simple program it is easy to find out that this doesn't have a memory leak. the way the program works is that it fetches a thousand rows from the Database, writes them to a file using Streams and then goes and fetches the next thousand rows. The size of each row varies but none of the rows is huge. On taking a dump while the program is running the older string are easily seen on the heap. These String in heap are unreachable which means they are waiting to get Garbage collected. I also believe that the GC doesn't necessarily run during the execution of this program which leaves String's in the heap longer than they should.

I think the solution would be to use long Char Arrays(or Stringbuffer) instead of using String objects to store the lines that are returned by the DB. The assumption is that I can overwrite the contents of a Char Array which means the same Char Array can be used across multiple iterations without having to allocate new Space each time.

Pseudocode :

1. Create an Array of Arrays using new char[1000][1000];
2. Fill the thousand rows from DB to the Array.
3. Write Array to File.
4. Use the same Array for next thousand rows

If the above pseudocode fixes my problem then in reality the Immutable nature of the String class hurts the Java programmer as there is no direct way to claim the space used up by a String even though the String is no longer in use.

Are there any better alternatives to this problem ?

PS : I didn't do a static analysis alone. I used yourkit profiler to test a heap dump. The dump clearly says 96% of the Strings have NO GC Roots which means they are waiting to get Garbage collected. Also I don't use Substring in my code.

Answer 1

Immutability of the class String has absolutely nothing to do with OutOfMemoryError . Immutability means that it cannot ever change, only that.

If you run out of memory, it is simply because the garbage collector was unable to find any garbage to collect .

In practice, it is likely that you are holding references to way too many Strings in memory (for instance, do you have any kind of collection holding strings, such as List, Set, Map?). You must destroy these references to allow the garbage collector to do its job and free up some memory.

Answer 2

The simple answer to this question is 'no'. I suspect you're hanging onto references longer than you think.

Are you closing those streams properly ? Are you intern() ing those strings. That would result in a permanent copy being made of the string if it doesn't exist already, and taking up permgen space (which isn't collected). Are you taking substring() of a larger string ? Strings make use of the flyweight pattern and will share a character array if created using substring() . See here for more details.

You suggest that garbage collection isn't running. The option -verbose:gc will log the garbage collections and you can see immediately what's going on.

Answer 3

The only thing about strings which can cause an OutOfMemoryError is if you retain small sections of a much larger string. If you are doing this it should be obvious from a heap dump.

When you take a heap dump I suggest you only look at live objects, in which case any retained objects you don't need is most likely to be a bug in your code.