在循环中实例化对象而不会出现 OutOfMemoryError：Java 堆空间

Question

I have to make a function that receives as en entry a List<PDXObjectImage> list and creates a small icon for each of these elements and store them in a JTable.

Now I found a way to create icons from a PDXObjectImage without loading the whole Image so that my program does not throw OutOfMemoryError: Java heap space :

for(int k=0;k<list.size();k++)
{
    ByteArrayOutputStream output = new ByteArrayOutputStream();
    list.get(k).write2OutputStream(output);
    ByteArrayInputStream bais = new ByteArrayInputStream(output.toByteArray());
    ImageInputStream iis = ImageIO.createImageInputStream(bais);
    Iterator iter = ImageIO.getImageReaders(iis);
    if (iter.hasNext()) {
        ImageReader reader = (ImageReader) iter.next();
        reader.setInput(iis, true, true);
        ImageReadParam params = reader.getDefaultReadParam();
        params.setSourceSubsampling(2.0, 2.0, 0, 0);
        BufferedImage img = reader.read(0, params);
        ImageIcon imageIcon = new ImageIcon(img);
        model.addRow(new Object[]{imageIcon});
    }
}

I managed to avoid the OutOfMemoryError: Java heap space for a small amount of pictures in the list using readers instead of loading the images as BufferedImage each time. Unfortunately, I still get this error when more than 84 elements are stored in the list.

I used jvisualvm to see what objects took all the heap space and I found out it was byte[] objects(around 85%).

The problem is clearly located where I create all the streams to get the iconImage . The thing is I don't know any ways of getting a ImageInputStream without having to create new streams each time.

I tried to avoid the problem by generating all the streams in a single function :

private ImageInputStream fct(PDXObjectImage img) throws IOException{
    ByteArrayOutputStream output = new ByteArrayOutputStream();
    img.write2OutputStream(output);
    ByteArrayInputStream bais = new ByteArrayInputStream(output.toByteArray());
    return ImageIO.createImageInputStream(bais);
}

thinking that java would automatically delete objects when it reach the end of the scope.

I tried adding the following in any order possible at the end of each loop :

output.reset();
output.flush();
bais.reset();
bais.close();
iis.flush();
output=null;
bais=null;
iis=null;
System.gc();

I also tried to instantiate the streams outside the function's scope, but there is no way of setting a ByteArrayInputStream from a byte[] without using the new keyword, thus creating a new object.

I still get the same error, nothing works.

I read some posts on Statements and ResultSets but I not found them relevant. (Maybe I am wrong)

If someone has any idea of how I could avoid this error, I would be very gratefull.

Thank you

EDIT:

I have modified my code so that I get the following :

for(int k=0;k<list.size();k++)
{
    list.get(k).write2OutputStream(cbb.getOutputStream());
    ImageInputStream iis = ImageIO.createImageInputStream(cbb.getInputStream());
    Iterator iter = ImageIO.getImageReaders(iis);
    if (iter.hasNext()) {
        ImageReader reader = (ImageReader) iter.next();
        reader.setInput(iis, true, true);
        BufferedImage img = reader.read(0, null);
        ImageIcon imageIcon = new ImageIcon(img);
        model.addRow(new Object[]{imageIcon});
    }
}

I have also added a listener to the reader so that it prints out the percentage of the reading done. It always goes up to 84.2% and stops.

Does anyone knows how can this be possible?

Answer 1

Use PipedInputStream/PipedOutputStream or CircularByteBuffer to channel the bytes written directly to the input stream. This way you will not have to create intermediate streams and waste memory.

Take a look at this post :

http://ostermiller.org/convert_java_outputstream_inputstream.html

Rewriting fct method using CircularByteBuffer :

private ImageInputStream fct(PDXObjectImage img) throws IOException{
CircularByteBuffer cbb = new CircularByteBuffer(CircularByteBuffer.INFINITE_SIZE);
img.write2OutputStream(cbb.getOutputStream());
return ImageIO.createImageInputStream(cbb.getInputStream());
}

You can also use a multi-threaded approach where you are writing bytes in one thread and reading on another. Thus, writing/reading can occur concurrently optimizing the CPU usage and memory utilization.

Note: com.Ostermiller.util.CircularByteBuffer is not a standard Java API. But the source is freely available

Answer 2

1) There's nothing in your code that suggests that loaded image is compressed anyhow to "small icon", except in Iterator iter = ImageIO.getImageReaders(iis); line. Can you confirm image is compressed indeed? Otherwise, it may be the simple case of not enough RAM allocated to JVM heap;

2) Even if image is compressed to smaller byte volume, you may still have insufficient RAM allocated to JVM. Or your compressed image may have internal reference to uncompressed one (I'm not familiar with API used, so I can't tell for sure), resulting in larger image being retained in JVM heap. Using memory profiler, take a look how much memory is occupied on each iteration of loop and if it decreseas over the time due to GC. Then divide total heap by this figure, and you may have an idea how many icons you can load w/o getting OutOfMemory .

3) Statements and ResultSets have nothing to do with Java image processing, these are about working with relational databases.

Answer 3

You are only halving the image size in both dimensions (with the subsampling parameters). Creating an icon should scale the image so that the images larger dimension just matches the display size of the icon.

You need first to determine the image dimensions, then calculate the proper target size, then read the image with subsampling.

If your icon size is (for example) 100 by 100 pixels, the icon should be exactly 100 pixels in its larger dimension.