How can I detect if a subprocess is hung because its output buffer is full?

Question

Consider the following Bash script and Java program:

$ cat kb.sh
#!/bin/bash

# Prints $1 KB - fold adds a \n
tr '\0' '=' < /dev/zero | fold -w 1023 | head -n ${1:-10}

$ cat Demo.java 
import java.util.concurrent.TimeUnit;

class Demo {
  public static void main(String[] args) throws Exception {
    Process p = Runtime.getRuntime()
        .exec("/tmp/kb.sh " + (args.length > 0 ? args[0] : ""));
    if (p.waitFor(10, TimeUnit.SECONDS)) {
      System.out.println("Process terminated");
    } else {
      System.err.println("Process did not terminate");
      p.destroy();
      System.exit(1);
    }
  }
}

The Demo class starts kb.sh as a subprocess, and expects it to terminate quickly. kb.sh , for its part, outputs (presumably quickly), some number of KBs of data. We can verify that it runs quickly in practice:

$ time /tmp/kb.sh 10000 | wc
  10000   10000 10240000

real    0m0.398s
user    0m0.178s
sys 0m0.030s

When we run the Demo class however we see different behavior:

$ java -cp . Demo 64
Process terminated

$ java -cp . Demo 65
Process did not terminate

If we attempt to print ~65KB it hangs. I know why - Process is buffering the subprocess' output and when its buffer gets full the subprocess blocks until some data is read out of the buffer via Process.getInputStream() . If you added a call to ByteStreams.exhaust(p.getInputStream()); before p.waitFor() the process would always terminate successfully.

My question is, is there any way in Java to detect when a subprocess is being blocked like this? I fear the answer may be "not without reflection", since I don't see any such mechanism in any relevant APIs, but I could be missing something.

To forestall the inevitable "Why do you want to do this?", I'm writing a diagnostic utility to detect this in existing Process instances as it's an ongoing (and nefarious) source of bugs. I don't want to manipulate the Process or do anything destructive, I simply want to detect when the process has been stalled due to a full buffer so I can alert the caller.

NB: OS-dependent solutions, such as inspecting the output of ps , would be acceptable, but obviously aren't as ideal as a Java-only solution.

Answer 1

You don't have to detect it. You have to consume all its output, from both the standard output and standard errors. If you have code that doesn't do that, fix it.

Answer 2

Short answer: there is no way to detect whether a full buffer is the cause of a hung subprocess.

Longer answer: The Java I/O stream APIs do not provide any way to determine the state of a buffered stream. You can not determine whether a buffer is full. Worse, you can't even know how much space is available in the buffer, so it's difficult, if not impossible ,to determine whether the next write() operation will block or not. And, of course, once it's blocked, it doesn't respond to anything.

Short of requiring the child process to produce or respond to "heartbeat" pings to prove it's alive and not hung -- and once it's hung there's no way to know why -- there's not much you can do to proactively, or reactively, deal with full buffered streams other than reading them.