How to randomly communicate with a process without freezing the GUI in Java Swing?

Question

I'm building a chess GUI application whose job is to display the board and the pieces and prevent illegal moves from being entered.

It should also have features that involve communication with a chess engine (eg stockfish). This is what I'm struggling with right now. The chess engine is an exe file that is accessed using ProcessBuilder:

Process chessEngineProcess = new ProcessBuilder(chessEngineUrl).start();

InputStream processInputStream = chessEngineProcess.getInputStream();
OutputStream processOutputStream = chessEngineProcess.getOutputStream();

BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(processOutputStream));
BufferedReader reader = new BufferedReader(new InputStreamReader(processInputStream));

I want to send strings (commands in the UCI protocol) to the engine to which it responds by continuously outputting text for a few seconds or longer. This hangs up the GUI. I need to update a textArea (in real-time) in the GUI based on the engine's output. This wouldn't be a one-off type of operation. I would like to randomly do this (send a command and update GUI in real time) whenever certain GUI events happen (eg user makes a move).

I know that I need to do the stream reading in another thread and I know about SwingWorker but I simply can't get it to work properly.

What I tried: Since the stream reading is a blocking operation (we keep waiting for output from the engine) the stream reading thread never terminates.

With that in mind, I tried creating a class that extends SwingWorker<Void, String> and sets up and contains the chessEngineProcess (as well as its stream reader and writer) as a private member variable. I implemented the doInBackground and process methods. I also had a public method in this class for sending a command to the engine.

public void sendCommandToEngine(String command) {
        try {
            writer.write(command + '\n');
            writer.flush();
        } catch (IOException e) {
            JOptionPane.showMessageDialog(null, e.getMessage());
        }
    }

I do the stream reading in the doInBackground and then publish the output and update the GUI in the process method.

This results in very strange behavior when I send commands to the engine from my GUI classes (eg from event listeners). The displayed output is (sometimes partly and sometimes entirely?) wrong and often I get exceptions thrown.

I am at a loss and very desperate so please help! This is a very important project. Feel free to suggest any solution that you think would work!

EDIT: I get a null pointer exception with the following stack trace:

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
    at Moves.Move.isMovePossible(Move.java:84)
    at Moves.Move.executeMove(Move.java:68)
    at gui.ChessBoard.performEngineMove(ChessBoard.java:328)
    at gui.MainFrame.receiveEnginesBestMove(MainFrame.java:180)
    at gui.EngineWorker.process(EngineWorker.java:91)
    at javax.swing.SwingWorker$3.run(SwingWorker.java:414)
    at sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112)
    at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(SwingWorker.java:832)
    at sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112)
    at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(SwingWorker.java:842)
    at javax.swing.Timer.fireActionPerformed(Timer.java:313)
    at javax.swing.Timer$DoPostEvent.run(Timer.java:245)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Some details: Basically I have a "MainFrame" class that is a JFrame that contains all my GUI elements. This is where I add event listeners to my components. In certain event listeners, I call sendCommandToEngine . This will start the blocked doInBackground as the engine starts sending responses.

The process method can then call performEnginesMove on the chessBoard (which is a MainFrame component displaying the chess board) if it detects that a "best move" was output by the engine.

The performEnginesMove function checks if the move is valid (possible) and then makes the move on the board (with the help of the Move class).

For some reason, this doesn't work right.

Answer 1

I built a delegate for the Process and ProcessBuilder classes to show how the rest of the code should be used. I call those classes GameEngineProcess and GameEngineProcessBuilder respectively.

GameEngineProcess is creating the responses, which are simple String s to be appended directly into the JTextArea of the player's GUI. It actually extends Thread to let it run asynchronously. So the implementation of this specific class is not what you are asking for, but it is used to simulate the Process class. I added some delay in the responses of this class to simulate the time needed by the engine to generate them.

Then there is the custom class OnUserActionWorker which extends SwingWorker and does asynchronously what you are asking for: it receives the responses from the engine process and forwards them to the GUI which updates its JTextArea . This class is used once per engine request, ie we create and execute a new instance of this class for every request the user creates while interacting with the GUI. Note that this does not mean the engine is closed and reopened for each request. The GameEngineProcess is started once and then remains running for the whole game uptime.

I am assuming you have a means of telling whether a single engine request has all its responses completed. For the sake of simplicity in this code I wrote, there exists a message (of type String ) which is written each time at the process stream to indicate the end of the responses per request. This is the END_OF_MESSAGES constant. So this lets the OnUserActionWorker know when to terminate receiving responses, so the next instance of it will be later created for each new request.

And finally there is the GUI, which is a JFrame consisting of a JTextArea and a grid of buttons the player can interact with and send a request-command to the engine depending on the button pressed. Again I am using String s as the commands but I'm guessing this is probably what you will need too in this case.

Follows the code:

import java.awt.BorderLayout;
import java.awt.Dimension;
import java.awt.Font;
import java.awt.GridLayout;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.util.List;
import java.util.Objects;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JOptionPane;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.SwingWorker;

public class Main {

    //Just a simple 'flag' to indicate end of responses per engine request:
    private static final String END_OF_MESSAGES = "\u0000\u0000\u0000\u0000";

    //A class simulating the 'ProcessBuilder' class:
    private static class GameEngineProcessBuilder {
        private String executionCommand;

        public GameEngineProcessBuilder(final String executionCommand) {
            this.executionCommand = executionCommand;
        }

        public GameEngineProcessBuilder command(final String executionCommand) {
            this.executionCommand = executionCommand;
            return this;
        }

        public GameEngineProcess start() throws IOException {
            final GameEngineProcess gep = new GameEngineProcess(executionCommand);
            gep.setDaemon(true);
            gep.start();
            return gep;
        }
    }

    //A class simulating the 'Process' class:
    private static class GameEngineProcess extends Thread {
        private final String executionCommand; //Actually not used.
        private final PipedInputStream stdin, clientStdin;
        private final PipedOutputStream stdout, clientStdout;

        public GameEngineProcess(final String executionCommand) throws IOException {
            this.executionCommand = Objects.toString(executionCommand); //Assuming nulls allowed.

            //Client side streams:
            clientStdout = new PipedOutputStream();
            clientStdin = new PipedInputStream();

            //Remote streams (of the engine):
            stdin = new PipedInputStream(clientStdout);
            stdout = new PipedOutputStream(clientStdin);
        }

        public OutputStream getOutputStream() {
            return clientStdout;
        }

        public InputStream getInputStream() {
            return clientStdin;
        }

        @Override
        public void run() {
            try {
                final BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(stdout));
                final BufferedReader br = new BufferedReader(new InputStreamReader(stdin));
                String line = br.readLine();
                while (line != null) {
                    for (int i = 0; i < 10; ++i) { //Simulate many responses per request.
                        Thread.sleep(333); //Simulate a delay in the responses.
                        bw.write(line + " (" + i + ')'); //Echo the line with the index.
                        bw.newLine();
                        bw.flush();
                    }
                    bw.write(END_OF_MESSAGES); //Indicate termination of this particular request.
                    bw.newLine();
                    bw.flush();
                    line = br.readLine();
                }
                System.out.println("Process gracefull shutdown.");
            }
            catch (final InterruptedException | IOException x) {
                System.err.println("Process termination with error: " + x);
            }
        }
    }

    //This is the SwingWorker that handles the responses from the engine and updates the GUI.
    private static class OnUserActionWorker extends SwingWorker<Void, String> {
        private final GameFrame gui;
        private final String commandToEngine;

        private OnUserActionWorker(final GameFrame gui,
                                   final String commandToEngine) {
            this.gui = Objects.requireNonNull(gui);
            this.commandToEngine = Objects.toString(commandToEngine); //Assuming nulls allowed.
        }

        //Not on the EDT...
        @Override
        protected Void doInBackground() throws Exception {
            final BufferedWriter bw = gui.getEngineProcessWriter();
            final BufferedReader br = gui.getEngineProcessReader();

            //Send request:
            bw.write(commandToEngine);
            bw.newLine();
            bw.flush();

            //Receive responses:
            String line = br.readLine();
            while (line != null && !line.equals(END_OF_MESSAGES)) {
                publish(line); //Use 'publish' to forward the text to the 'process' method.
                line = br.readLine();
            }

            return null;
        }

        //On the EDT...
        @Override
        protected void done() {
            gui.responseDone(); //Indicate end of responses at the GUI level.
        }

        //On the EDT...
        @Override
        protected void process(final List<String> chunks) {
            chunks.forEach(chunk -> gui.responsePart(chunk)); //Sets the text of the the text area of the GUI.
        }
    }

    //The main frame of the GUI of the user/player:
    private static class GameFrame extends JFrame implements Runnable {
        private final JButton[][] grid;
        private final JTextArea output;
        private BufferedReader procReader;
        private BufferedWriter procWriter;

        public GameFrame(final int rows,
                         final int cols) {
            super("Chess with remote engine");

            output = new JTextArea(rows, cols);
            output.setEditable(false);
            output.setFont(new Font(Font.MONOSPACED, Font.ITALIC, output.getFont().getSize()));

            final JPanel gridPanel = new JPanel(new GridLayout(0, cols));

            grid = new JButton[rows][cols];
            for (int row = 0; row < rows; ++row)
                for (int col = 0; col < cols; ++col) {
                    final JButton b = new JButton(String.format("Chessman %02d,%02d", row, col));
                    b.setPreferredSize(new Dimension(b.getPreferredSize().width, 50));
                    b.addActionListener(e -> sendCommandToEngine("Click \"" + b.getText() + "\"!"));
                    gridPanel.add(b);
                    grid[row][col] = b;
                }

            final JScrollPane outputScroll = new JScrollPane(output);
            outputScroll.setPreferredSize(gridPanel.getPreferredSize());

            final JPanel contents = new JPanel(new BorderLayout());
            contents.add(gridPanel, BorderLayout.LINE_START);
            contents.add(outputScroll, BorderLayout.CENTER);

            super.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
            super.getContentPane().add(contents);
            super.pack();
        }

        //Utility method to enable/disable all the buttons of the grid at once:
        private void gridSetEnabled(final boolean enabled) {
            for (final JButton[] row: grid)
                for (final JButton b: row)
                    b.setEnabled(enabled);
        }

        //This is the method that sends the next request to the engine:
        private void sendCommandToEngine(final String commandToEngine) {
            gridSetEnabled(false);
            output.setText("> Command accepted.");
            new OnUserActionWorker(this, commandToEngine).execute();
        }

        public BufferedReader getEngineProcessReader() {
            return procReader;
        }

        public BufferedWriter getEngineProcessWriter() {
            return procWriter;
        }

        //Called by 'SwingWorker.process':
        public void responsePart(final String msg) {
            output.append("\n" + msg);
        }

        //Called by 'SwingWorker.done':
        public void responseDone() {
            output.append("\n> Response finished.");
            gridSetEnabled(true);
        }

        @Override
        public void run() {
            try {
                //Here you build and start the process:
                final GameEngineProcess proc = new GameEngineProcessBuilder("stockfish").start();

                //Here you obtain the I/O streams:
                procWriter = new BufferedWriter(new OutputStreamWriter(proc.getOutputStream()));
                procReader = new BufferedReader(new InputStreamReader(proc.getInputStream()));

                //Finally show the GUI:
                setLocationRelativeTo(null);
                setVisible(true);
            }
            catch (final IOException iox) {
                JOptionPane.showMessageDialog(null, iox.toString());
            }
        }
    }

    public static void main(final String[] args) {
        new GameFrame(3, 3).run(); //The main thread starts the game, which shows the GUI...
    }
}

And finally, another important assumption I made is that when the user interacts with the GUI, the GUI blocks input (but keeps on responding to other events). This prevents the user to have more than one active requests to the engine simultaneously. By blocking input I simply mean that when you click on a button, first all buttons are disabled and then the command is sent to the engine. The buttons are all then reenabled when all the responses for the latest made request finish.

If you need to have more than one requests possible at the same time to a single engine then you will probably need to synchronize the access of some of the GUI's methods and also make sure that each OnUserActionWorker can distinguish its responses from the others. So that would be a different story, but let me know if this is what you want instead.

To test the responsiveness of the EDT while the responses are being received you can for example simply resize the window with the mouse while the (ten) responses are still being received, or just notice that the responses are printed into the JTextArea at real time.

Hope it helps.