How can I fix this memory leak?

Question

import java.io.*;
import java.util.HashMap;
import java.util.logging.Logger;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;

public class ArchiveLoader {

    private static final Logger logger = Logger.getLogger(Landing.class.getName());

    private final String PREFIX = ".class";
    private final byte[] BUFFER = new byte[1024];

    private File archive;

    private HashMap<String, byte[]> classMap = new HashMap<>();

    public ArchiveLoader(String archivePath) throws IOException {
        this.archive = new File(archivePath);
    }

    public void load() throws IOException {
        FileInputStream fis = new FileInputStream(archive);
        loadStream(fis);
        fis.close();
    }

    private void loadStream(InputStream inputStream) throws IOException {
        if (archive.canRead()) {
            if (classMap.size() == 0) {
                ZipInputStream zis = new ZipInputStream(inputStream);

                ZipEntry entry;
                while ((entry = zis.getNextEntry()) != null) {
                    String name = entry.getName();
                    if (name.toLowerCase().endsWith(PREFIX)) {
                        name = name.substring(0, name.indexOf(PREFIX));
                        ByteArrayOutputStream bos = new ByteArrayOutputStream();
                        int read;
                        while ((read = zis.read(BUFFER, 0, BUFFER.length)) != -1) {
                            bos.write(BUFFER, 0, read);
                        }
                        zis.closeEntry();
                        bos.close();
                        classMap.put(name, bos.toByteArray());
                    }
                }

                inputStream.close();
                zis.close();
                logger.info("Loaded " + classMap.size() + " classes.");
            } else {
                logger.info("Archive has already been loaded!");
            }
        } else {
            throw new IOException("Could not read the JAR archive.");
        }
    }

    public void clear() {
        classMap.clear();
        System.out.println(classMap.size());
        classMap = null;
        logger.info("`enter code here`Cleared the ArchiveLoader.");
    }

}

• Before I load my JAR archive the memory usage is around 14mb.
• When I load a jar file with that class the memory usage goes to around 210mb.
• When I call clear the memory usage does not decrease.

How can I decrease the memory usage again?

  for (int i = 0; i < 10; i++) { ArchiveLoader archiveLoader = new ArchiveLoader(FileManager.getClientLocation()); archiveLoader.load(); archiveLoader.clear(); } 

When I ran this, the memory usage went all the way up to 660mb and then decreased to 526mb. From that point it wouldn't stop dropping anymore.

Answer 1

Why are you clearing the map in the first place? That would only sort-of make sense if you intend to re-use the archive loader instance for another archive. Why not just instantiate another instance if you want to do that and let this one be cleared automatically by the GC once you have no reference to it anymore? It seems you're trying to do too much memory-management than needed when coding in Java. I don't think you have a memory leak in the first place, like Boris said the memory usage won't immediately drop down. If you are actually running out of memory after loading archives over and over, then you'll know you have a memory leak. Otherwise that analysis isn't this simple.

Answer 2

Q: Is there a memory leak?

I'm not convinced you have a memory leak at all. In fact, this looks how I would expect a non-leaky program to behave if the amount of reachable data was highly variable.

Firstly, you are looking at the memory usage as reported by the OS. This includes all memory used by the the JVM including various out-of-heap resources such as native libraries and stacks. It also includes heap overheads such as evacuated spaces; eg memory that counts towards the heap's "free space".

To determine if you have a real memory leak (in the Java heap), you need to see the minimum and maximum heap usage over a period of time. Specifically, you need to get hold of the "used" and "free" values before and after the GC is run .... over a number of GC cycles. If there is a clear upwards trend of these values (at those points) over time, then you have a problem.

Q: How do you get hold of that information?

The simple way is to use the Oracle visualvm tool as described here. The memory usage graph will look like a sawtooth with the "peaks" and "valleys" corresponding to a garbage collection. What you are looking for is a long-term upwards trend in the height of the peaks and valleys.

If visualvm provides (real) evidence of a leak then it also has tools to help you track them down.

Q: So why does Windows say you are using so much memory?

Well, basically you ARE using that memory. The JVM asks the OS for enough memory to make the heap as big as it needs to be to hold all of your objects. In your case, the "demand" is fluctuating between two extremes. The problem for the JVM is as follows:

• It does not know what your application is going to do. It doesn't know how much memory your application is going to ask for, when it is going to release it and whether it is going to ask for it back.

• It only "does something" when it runs the GC, and that only happens when a "space" gets full, or close to full. (That is unlikely to correspond to your clear() calls.)

• There are significant costs for the JVM in giving unused memory back to the OS. Objects need to be moved around so that "spaces" can be resized without fragmenting the address space.

So what this means is that if you have an application with a "bursty" memory demand profile, the JVM is likely to size the heap to hold the maximum demands, and leave it at that level in the long term.

This is not to say that the JVM never gives memory back. Depending on the JVM heap tuning parameters, if the JVM sees that the heap is too large after a number of GC cycles, it will reduce it by giving memory back. However, it does this in conservative / reluctant fashion. The reasons for the reluctance are that:

• Garbage collection is more efficient if the heap is large.

• Growing the heap (again) is an overhead that it wants to avoid.

Q: Should you run System.gc() ?

No! No! No!

It is possible to force the GC to run (as above), but it is much better for system performance to leave it to the JVM to decide when it is efficient to to it.

Besides, you have no guarantee that the running the GC will lead to the JVM giving any memory back to the OS ... if your goal is to reduce memory usage at the system level.

Q: How do I use as little (memory) resources as possible.

1. Rewrite your application is a non-managed language like C or C++ and implement your own memory management.

2. Don't cache JAR file content in memory like that.

Answer 3

I hope the other answers provided reassurances wrt memory leakage, and JVM behaviour.

Still your program might leak - on exceptions (shown however in the logs). A minor thing however.

Use try-with-resources to prevent resource leakage on exceptions and such.

try (FileInputStream fis = new FileInputStream(archive)) {
    loadStream(fis);
} // Always closes fis.

HOWEVER, in your case the code wraps the FileInputStream in a ZipInputStream, and the code closes three times, where generally one would only close the ZipInputStream.

A redesign, with loadStream() using this.archive would seem best, closing the ZipInputStream using try-with-resources.