Help running Java runtime.exec() on multiple threads

Question

In my program, I need to run a external command in a Ubuntu environment (ntpdate) using java. Currently my code looks like this:

    Runtime rt = Runtime.getRuntime();

    byte[] readBuffer = new byte[131072];
    // Exec a process to do the query
    Process p = null;
    try {
        p = rt.exec("ntpdate -q " + ip);
    } catch (Exception ex) {
        ex.printStackTrace();
    }

    if(p!= null){
        try {
            Thread.sleep(1000);
        } catch (Exception e) {
        }

        // Read the input stream, copy it to the file
        InputStream in = p.getInputStream();


        try {
            int count = 0, rc;
            while ((rc = in.read(readBuffer, count, readBuffer.length - count)) != -1) {
                count += rc;
                if (count >= readBuffer.length) {
                    p.destroy();
                    break;
                }
            }
            p.destroy();
            result = processOutput(readBuffer, count);

        } catch (IOException ex) {
            ex.printStackTrace();
        }
        p.destroy();

This code need to be ran simultaneously on multiple threads in order to maximize performance (I need to test a list of 1.000.000 addresses using ntpdate). However, it runs very slowly, barely consuming machine processing. What am I doing wrong? How could I make this more efficient?

The same problem arises when trying to execute "dig" using .exec(), so I doubt it is because of the specific program being called. Is there some restriction in using Runtime.exec() in a multi Threaded environment?

Answer 1

Is Java the most appropriate approach here? Perhaps this would be better in a shell script, which calls ntpdate in the background multiple times? I'm not sure what benefit you're getting from this code snippet by doing this in Java.

What are you doing with the InputStream from the process?

---

A bash script could do this like:

for ip in #...IP list
do
  ntpdate -q $ip > $ip.txt &
done

Answer 2

Why are you waiting for 1 second at each time ?

try {
    Thread.sleep(1000);
} catch (Exception e) {
}

This will do nothing but slowing the execution of your application.

Answer 3

Not sure why it's slow but you need to do a lot more to close your resources. Runtime.exec() needs quite a bit of care and attention to avoid hang-ups and leaking of file descriptors.

http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html

Answer 4

Are you sure the issue isn't ntpdate? If ntpdate is just sitting there waiting for a server response and has a large timeout value, then your application is going to sit there too.

Try calling ntpdate with a timeout of 0.2 and see if it makes a difference.

Also, as you're opening streams in your code, you definitely want to explicitly .close() them when you're done. Otherwise it might not happen until a GC which could be a very long time away.

Answer 5

I think I found the solution, and that is that there is no solution using java's Runtime.exec(). The problem seems to be that all calls to start a process are synchronized. Indeed, if you start each process alone (via synchronization) you get the exact same result of starting all processes together.

Are there any alternatives to exec? Otherwise, I will need to get some solution without linux's ntpdate...

Answer 6

I notice that both of the commands you tried involve network round-trips. How is the speed if you call something like echo or cat instead?