Java: after jar is removed, class is still loaded

Question

I was trying to reproduce a bug where, a jar is updated (via rsync on a linux box) and then a NoClassDefFoundError was thrown. The updated jar was unchanged, but I was thinking about the fact the file was transferring while the class was loading...

I'm now trying to reproduce the bug.

My application start with a classpath of only one jar ( /opt/test/myjar.jar )

The others jar are inside a directory on the same path of myjar.jar ( /opt/test/lib/mylib.jar ).

The library is registered into the myjar.jar META-INF/MANIFEST.MF with this text

Manifest-Version: 1.0
Built-By: FB
Class-Path: lib/mylib.jar

Now I scripted some code for waiting some seconds, then load some class with Class.forName("mylib.MyClass") .

Then I will setup the folder, start the java runtime and then delete the lib/mylib.jar file, and wait for the Class.forName to fail.

And the code was running fine. I was expecting a NoClassDefFoundError . Then I rerun the code, and a NoClassDefFoundError was thrown.

Then I readded the mylib.jar to the lib directory, rerun, all ok.

Then I rerun the code with -verbose:class , deleted the lib/mylib.jar and then this log appeared.

[Loaded mylib.MyClass from file:/opt/test/lib/mylib.jar`]

So the class loading was happening after the jar deletion. I don't understand why this work. And no other classes were loaded from lib/mylib.jar before.

Jdk used is OpenJDK Runtime Environment Corretto-8.302.08.1 (build 1.8.0_302-b08)

I don't understand how the JVM can load a class from a file I just deleted. I think maybe the JVM cache those files somewhere (mabye cause they are registered inside the MANIFEST.MF ).

Have anyone an idea of this behavior??

Ps. I tested this exact procedure, but with real jar and classes. If no one have an idea on the why, I can build an test project.

Answer 1

You are using a system without mandatory file locking. If you tried the same under Windows, for example, you couldn't do neither, overwrite nor delete the.jar file.

The jar files on the class path are opened when the JVM starts and kept open during the runtime. We can demonstrate the behavior using ordinary file operations:

Path p = Files.createTempFile(Paths.get(System.getProperty("user.home")),"test",".tmp");
try(FileChannel ch = FileChannel.open(p,
                                 StandardOpenOption.READ, StandardOpenOption.WRITE)) {
  System.out.println("opened " + p);
  int rc = new ProcessBuilder("rm", "-v", p.toString()).inheritIO().start().waitFor();
  System.out.println("rm ran with rc " + rc);
  int w = ch.write(StandardCharsets.US_ASCII.encode("test data"));
  System.out.println("wrote " + w + " bytes into " + p);
  ch.position(0);
  ByteBuffer bb = ByteBuffer.allocate(w);
  do ch.read(bb); while(bb.hasRemaining());
  bb.flip();
  System.out.println("read " + bb.remaining() + " bytes, "
                   + StandardCharsets.US_ASCII.decode(bb));
}
System.out.println("closed, reopening");
try(FileChannel ch = FileChannel.open(p,
                                 StandardOpenOption.READ, StandardOpenOption.WRITE)) {
  System.out.println("opened " + p);
}
catch(IOException ex) {
  System.out.println("Reopening " + p + ": " + ex);
}

prints something like

opened /home/tux/test722563514590118445.tmp
removed '/home/tux/test722563514590118445.tmp'
rm ran with rc 0
wrote 9 bytes into /home/tux/test722563514590118445.tmp
read 9 bytes, test data
closed, reopening
Reopening /home/tux/test722563514590118445.tmp: java.nio.file.NoSuchFileException: /home/tux/test722563514590118445.tmp

demonstrating that after removing, we can still write and read data from the already opened file, as only the entry has been removed from the directory. The JVM is now operating on a file without a name. But as soon as this filehandle has been closed, trying to open it again will fail, as now it is really gone.

---

Overwriting the file, however, is a different thing. When opening the existing file, we access the same file and make changes perceivable.

So

Path p = Files.createTempFile(Paths.get(System.getProperty("user.home")),"test",".tmp");
try(FileChannel ch = FileChannel.open(p,
                                 StandardOpenOption.READ, StandardOpenOption.WRITE)) {
  System.out.println("opened " + p);
  int w = ch.write(StandardCharsets.US_ASCII.encode("test data"));
  System.out.println("wrote " + w + " bytes into " + p);
  int rc = new ProcessBuilder("cp", "/proc/self/cmdline", p.toString())
      .inheritIO().start().waitFor();
  System.out.println("cp ran with rc " + rc);
  ch.position(0);
  ByteBuffer bb = ByteBuffer.allocate(w);
  do ch.read(bb); while(bb.hasRemaining());
  bb.flip();
  System.out.println("read " + bb.remaining() + " bytes, "
                   + StandardCharsets.US_ASCII.decode(bb));
}

produces something like

opened /home/tux/test7100435925076742504.tmp
wrote 9 bytes into /home/tux/test7100435925076742504.tmp
cp ran with rc 0
read 9 bytes, cp/proc/

Showing that the read operation on the already open file resulted in what cp wrote, partially of course, as the buffer was pre-sized to what the Java application wrote. This demonstrates how overwriting an open file can cause havoc when some data has been read already and the application tries to interpret the new data according to what it knows from the old version.

---

This leads to a solution for updating a jar file without crashing the already running JVM. Delete the old jar file first, which lets the JVM run with the already opened, now-private old file, before copying the new version to the same location. From the system's perspective, you have two different files then. The old one will cease to exist when the JVM terminates. JVMs started after the replacement will use the new version.