Get process output in real time with java

Question

I would like to print live output of a process in java; but the output is (almost) only available when the process stops execution.

In other words, when I run the same process with the command line, I get more frequent feedback (output) than when I execute it with Java.

I tested the following code to print the output every 250ms:

private static String printStream(BufferedReader bufferedReader) {
    try {
        String line = bufferedReader.readLine();
        if(line != null) {
            System.out.println(line);
        }
        return line;
    } catch (IOException ex) {
        ex.printStackTrace();
    }
    return null;
}

public static void sendProcessStreams(Process process, SendMessage sendMessage) {
    String[] command = { "/bin/bash", "-c", "custom_process"};
    ProcessBuilder processBuilder = new ProcessBuilder(command);
    processBuilder.directory(new File("."));
    Process process = processBuilder.start();

    BufferedReader inputBufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()), 1);
    BufferedReader errorBufferedReader = new BufferedReader(new InputStreamReader(process.getErrorStream()), 1);
    System.out.println("Start reading process");
    Timer timer = new Timer();
    timer.scheduleAtFixedRate(new TimerTask() {
          @Override
          public void run() {
              String inputLine = printStream(inputBufferedReader);
              String errorLine = printStream(errorBufferedReader);
              if(inputLine == null && errorLine == null && !process.isAlive()) {
                  timer.cancel();
              }
          }
     }, 0, 250);
}

Yet it only prints two lines, and then waits for the process to end before printing everything else.

Is it possible to get more frequent feedback from an external process? If not, Why not?

Answer 1

Your code basically (as far as reading from the output) works correctly, the problem is surely for another reason, please build a minimal problem that is reproducible (eg. what is custom_process , why you use Timer when you can use the current thread, ...).

Anyway, here's an example reading in realtime the output:

final Process proc = new ProcessBuilder(
            "/bin/bash", "-c",
            "for i in `seq 1 10`; do echo $i; sleep $((i % 2)); done")
           .start();
try(InputStreamReader isr = new InputStreamReader(proc.getInputStream())) {
    int c;
    while((c = isr.read()) >= 0) {
        System.out.print((char) c);
        System.out.flush();
    }
}

With output:

1         (and wait one second)
2
3         (and wait one second)
4
5         (and wait one second)
6
7         (and wait one second)
8
9         (and wait one second)
10

Answer 2

it seems like you are dealing with multithreading-problems and not with getting the output of the process.

I just made this demo class that you can use:

CommandExecTest.java

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.util.ArrayList;

public class CommandExecTest {
    public static void main(String[] args) throws InterruptedException {

        String executable = "cmd";
        String[] commandParams = {"@ping -n 5 localhost","echo \"hello world\"","exit 123"};
        boolean passCommandsAsLinesToShellExecutableAfterStartup = true;
        AsyncExecutor asyncExecutor = new AsyncExecutor(executable, commandParams,passCommandsAsLinesToShellExecutableAfterStartup);
        System.out.println("x"+"/x\tsecs in main thread \t\t status:"+asyncExecutor.runstate+" of async thread that monitors the process");
        asyncExecutor.start();//start() invokes the run() method as a detached thread
        for(int i=0;i<10;i++) {
            // you can do whatever here and the other process is still running and printing its output inside detached thread
            Thread.sleep(1000);
            System.out.println(i+"/10\tsecs in main thread \t\t status:"+asyncExecutor.runstate+" of async thread that monitors the process");
        }

        asyncExecutor.join(); // main thread has nothing to do anymore, wait till other thread that monitor other process finishes as well
        System.out.println("END OWN-PROGRAMM: 0 , END OTHER PROCESS:"+asyncExecutor.processExitcode);
        System.exit(0);
    }
}

Runstate.java

public static enum Runstate {
    CREATED, RUNNING, STOPPED
}

AsyncExecutor.java

public static class AsyncExecutor extends Thread{
    private String executable;
    private String[] commandParams;
    public ArrayList<String> linesSoFarStdout = new ArrayList<>();
    public ArrayList<String> linesSoFarStderr = new ArrayList<>();
    public Runstate runstate;
    public int processExitcode=-1;
    private boolean passCommandsAsLinesToShellExecutableAfterStartup = false;
    public AsyncExecutor(String executable, String[] commandParams) {
        this.executable=executable;
        this.commandParams=commandParams;
        this.runstate=Runstate.CREATED;
        this.passCommandsAsLinesToShellExecutableAfterStartup=false;
    }
    /**
     * if you want to run a single-process with arguments use <b>false</b> example executable="java" commandParams={"-jar","myjarfile.jar","arg0","arg1"} 
     * <p>
     * if you want to run a shell-process and enter commands afterwards use <b>true</b> example executable="cmd" commandParams={"@ping -n 5 localhost","echo \"hello world\"","exit 123"} 
     * @param executable
     * @param commandParams
     * @param passCommandsAsLinesToShellExecutableAfterStartup
     */
    public AsyncExecutor(String executable, String[] commandParams, boolean passCommandsAsLinesToShellExecutableAfterStartup) {
        this.executable=executable;
        this.commandParams=commandParams;
        this.runstate=Runstate.CREATED;
        this.passCommandsAsLinesToShellExecutableAfterStartup=passCommandsAsLinesToShellExecutableAfterStartup;
    }
    @Override
    public void run() {
        this.runstate=Runstate.RUNNING;
        // 1 start the process
        Process p = null;
        try {
            if(passCommandsAsLinesToShellExecutableAfterStartup) {
                // open a shell-like process like cmd and pass the arguments/command after opening it
                // * example:
                // * open 'cmd' (shell)
                // * write 'echo "hello world"' and press enter
                p = Runtime.getRuntime().exec(new String[] {executable});
                PrintWriter stdin = new PrintWriter( p.getOutputStream());
                for( int i = 0; i < commandParams.length; i++) { 
                    String commandstring = commandParams[i];
                    stdin.println( commandstring);
                }
                stdin.close();
            }
            else {
                // pass the arguments directly during startup to the process
                // * example:
                // * run 'java -jar myexecutable.jar arg0 arg1 ...'
                String[] execWithArgs = new String[commandParams.length+1];
                execWithArgs[0] = executable;
                for(int i=1;i<=commandParams.length;i++) {
                    execWithArgs[i]=commandParams[i-1];
                }
                p = Runtime.getRuntime().exec( execWithArgs);
            }
            // 2 print the output
            InputStream is = p.getInputStream();
            BufferedReader br = new BufferedReader( new InputStreamReader( is));

            InputStream eis = p.getErrorStream();
            BufferedReader ebr = new BufferedReader( new InputStreamReader( eis));

            String lineStdout=null;
            String lineStderr=null;
            while(p.isAlive()) {
                Thread.yield(); // *
                // * free cpu clock for other tasks on your PC! maybe even add thread.sleep(milliseconds) to free some more
                // * everytime this thread gets cpu clock it will try the following codeblock inside the while and yield afterwards for the next time it gets cpu-time from sheduler
                while( (lineStdout = br.readLine()) != null || (lineStderr = ebr.readLine()) != null) {
                    if(lineStdout!=null) {
                        System.out.println(lineStdout);
                        linesSoFarStdout.add(lineStdout);
                    }
                    if(lineStderr!=null) {
                        System.out.println(lineStderr);
                        linesSoFarStderr.add(lineStderr);
                    }
                }
            }
            // 3 when process ends
            this.processExitcode = p.exitValue();
        }
        catch(Exception e) {
            System.err.println("Something went wrong!");
            e.printStackTrace();
        }
        if(processExitcode!=0) {
            System.err.println("The other process stopped with unexpected existcode: " + processExitcode);
        }
        this.runstate=Runstate.STOPPED;
    }
}