How to send data to a command line from a python script

Question

I am creating a program that plays chess using Stockfish. I intend to play in the terminal that handles the board and the commands. I want to be able to send data to the terminal by first getting it in python, but I cannot figure how to send data to the terminal from my searches.

As a simple example to start, I would like to do something like this:

os.startfile(path_to_executable)  # dont necessaryily need this, but I found I can run the .exe through python using this

command = "arbitrary string" 

while command != "end":
    command = input("Type command, type end to finish")  # get the command to run
    if command != "end":
        write_command_to_terminal(command)  # send the command to the terminal (Question I am asking)

def write_command_to_terminal(command: str):
    # fixme: how do I write to the other executable?

I am aware there are libraries built to utilize stockfish and other chess related things, however I intend on not using them and instead making it from scratch using the UCI protocol.

Answer 1

This problem isn't trivial because you can't use Popen.communicate() to do ongoing communication with a process. communicate() actually closes the stdin of the running process so you can only write to it once, which it seems like wouldn't work for you.

The following example program shows a way to do ongoing communication with a running process. It includes both the "client" code (what you hope to write) along with a stub of the "server" code (which in your case would presumably be a stockfish program).

from subprocess import PIPE, Popen
from sys import argv


def server():
    """Dummy 'other' process to show how this works"""
    while True:
        line = input()
        print(f"Got {line} from stdin in server")


def client():
    """
    Starts another process and sends arbitrary commands to it. Replace
    the command arguments to Popen to start the appropriate process.
    """
    terminal = Popen(["python3", __file__, "server"], stdin=PIPE)
    try:
        command = "arbitrary string"

        while command != "end":
            command = input("Type command, type end to finish: ")
            if command == "end":
                return

            write_command_to_terminal(command, terminal)
    finally:
        terminal.kill()


def write_command_to_terminal(command, terminal):
    """
    Writes the command to the terminal process.
    """
    terminal.stdin.write((command + "\n").encode())
    terminal.stdin.flush()


if __name__ == "__main__":
    if len(argv) == 1 or argv[1] == "client":
        client()
    elif argv[1] == "server":
        server()
    else:
        raise Exception("Invalid argument")

Answer 2

Here is a sample program to allow us to play a uci engine like stockfish from the command line.

At the prompt, if command starts with #, it is just a comment.

Code

from subprocess import Popen, PIPE, STDOUT


def command(p, command):
    p.stdin.write(f'{command}\n')
    
    
def play(efile):
    # start the engine, be sure to send quit command to terminate the program and the engine
    engine = Popen([efile], stdout=PIPE, stdin=PIPE, stderr=STDOUT, bufsize=0, text=True)
    
    while True:
        userinput = input('> ')
        line = userinput.rstrip()
        
        if line.startswith('go'):
            # go movetime 1000
            command(engine, line)
            for elines in iter(engine.stdout.readline, ''):
                eline = elines.strip()
                print(eline)
                if 'bestmove' in eline:
                    break

        elif line.startswith('#'):
            pass
        
        elif line == 'ucinewgame':
            command(engine, line)

        elif line == 'board':
            # Show board
            command(engine, 'd')
            for elines in iter(engine.stdout.readline, ''):
                eline = elines.strip()
                print(eline)
                if 'Checkers' in eline:
                    break

        elif line.startswith('setoption name'):
            command(engine, line)

        elif 'position ' in line:
            # position startpos moves e2e4
            command(engine, line)

        elif line == 'uci':
            command(engine, 'uci')
            for elines in iter(engine.stdout.readline, ''):
                eline = elines.strip()
                print(eline)
                if 'uciok' in eline:
                    break

        elif line == 'isready':
            command(engine, line)
            for elines in iter(engine.stdout.readline, ''):
                eline = elines.strip()
                print(eline)
                if 'readyok' in eline:
                    break

        elif line == 'quit':
            command(engine, line)
            break
            

# Start
efile = 'F:\\Chess\\Engines\\stockfish\\sf14\\sf14.exe'
play(efile)

Sample output

python play_sf.py
> # send uci command to see the engine id and author 
> uci
Stockfish 14 by the Stockfish developers (see AUTHORS file)
id name Stockfish 14
id author the Stockfish developers (see AUTHORS file)      

option name Debug Log File type string default
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 33554432
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 10 min 0 max 5000
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name UCI_LimitStrength type check default false
option name UCI_Elo type spin default 1350 min 1350 max 2850
option name UCI_ShowWDL type check default false
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
option name Use NNUE type check default true
option name EvalFile type string default nn-3475407dc199.nnue
uciok
> # check if engine is ready
> isready
readyok
> # ohh its ready
> # set the hash memory to 64 mb
> setoption name Hash value 64
> # setup the start position
> position startpos
> # let's see the board
> board

+---+---+---+---+---+---+---+---+  
| r | n | b | q | k | b | n | r | 8
+---+---+---+---+---+---+---+---+  
| p | p | p | p | p | p | p | p | 7
+---+---+---+---+---+---+---+---+  
|   |   |   |   |   |   |   |   | 6
+---+---+---+---+---+---+---+---+  
|   |   |   |   |   |   |   |   | 5
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 4
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 3
+---+---+---+---+---+---+---+---+
| P | P | P | P | P | P | P | P | 2
+---+---+---+---+---+---+---+---+
| R | N | B | Q | K | B | N | R | 1
+---+---+---+---+---+---+---+---+
a   b   c   d   e   f   g   h

Fen: rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
Key: 8F8F01D4562F59FB
Checkers:
> # we play white and our first move is e2e4
> position startpos moves e2e4
> # We let the engine search for 1sec or 1000 millisec
> go movetime 1000
info string NNUE evaluation using nn-3475407dc199.nnue enabled
info depth 1 seldepth 1 multipv 1 score cp -13 nodes 22 nps 22000 tbhits 0 time 1 pv e7e5
info depth 2 seldepth 2 multipv 1 score cp -3 nodes 46 nps 46000 tbhits 0 time 1 pv e7e5 a2a3
info depth 3 seldepth 3 multipv 1 score cp -1 nodes 116 nps 116000 tbhits 0 time 1 pv e7e5 a2a3 f8c5
info depth 4 seldepth 4 multipv 1 score cp -47 nodes 403 nps 201500 tbhits 0 time 2 pv g8f6 b1c3 e7e5 g1f3
info depth 5 seldepth 5 multipv 1 score cp -65 nodes 968 nps 322666 tbhits 0 time 3 pv e7e6 d2d4 d7d5 b1c3
info depth 6 seldepth 6 multipv 1 score cp -40 nodes 1706 nps 341200 tbhits 0 time 5 pv c7c6 d2d4 d7d5 e4e5
info depth 7 seldepth 7 multipv 1 score cp -32 nodes 2308 nps 384666 tbhits 0 time 6 pv c7c6 d2d4 d7d5 e4d5 c6d5 g1f3 c8g4
info depth 8 seldepth 9 multipv 1 score cp -37 nodes 5030 nps 558888 tbhits 0 time 9 pv e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4
info depth 9 seldepth 11 multipv 1 score cp -40 nodes 8039 nps 535933 tbhits 0 time 15 pv e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 c6e5
info depth 10 seldepth 15 multipv 1 score cp -36 nodes 16143 nps 597888 tbhits 0 time 27 pv e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e5d4 f1e1
info depth 11 seldepth 15 multipv 1 score cp -55 nodes 38538 nps 621580 tbhits 0 time 62 pv e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 f8b4 c2c3 b4c5 d4c6 b7c6
info depth 12 seldepth 16 multipv 1 score cp -37 nodes 64205 nps 668802 tbhits 0 time 96 pv c7c5 g1f3 b8c6 f1b5 e7e5 e1g1 d7d6 f1e1 g8f6 b1c3 c8g4
info depth 13 seldepth 16 multipv 1 score cp -32 nodes 101299 nps 693828 tbhits 0 time 146 pv c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 d8d7 e1g1 g7g6 c2c3 g8f6 f1e1 f8g7
info depth 14 seldepth 18 multipv 1 score cp -33 nodes 135174 nps 700383 tbhits 0 time 193 pv c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 d8d7 e1g1 g7g6 c2c3 g8f6 f1e1 f8g7 d2d4 c5d4 c3d4 b8c6
info depth 15 seldepth 21 multipv 1 score cp -33 nodes 209066 nps 718439 tbhits 0 time 291 pv c7c5 c2c3 g8f6 e4e5 f6d5 g1f3 b8c6 f1c4 d5b6 c4b5 d7d5 d2d4 c5d4 c3d4
info depth 16 seldepth 20 multipv 1 score cp -25 nodes 348508 nps 723045 tbhits 0 time 482 pv c7c5 c2c3 g8f6 e4e5 f6d5 d2d4 c5d4 g1f3 b8c6 c3d4 e7e6 f1c4 d7d6 e1g1 f8e7 d1e2 e8g8 b1c3
info depth 17 seldepth 22 multipv 1 score cp -26 nodes 456537 nps 720089 tbhits 0 time 634 pv c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 d8d7 e1g1 g8f6 f1e1 e7e6 c2c3 f8e7 d2d4 e8g8 d4d5 e6d5
info depth 18 seldepth 25 multipv 1 score cp -26 nodes 643841 nps 718572 tbhits 0 time 896 pv c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 d8d7 e1g1 e7e6 c2c3 d6d5 e4d5 d7d5 b1a3 b8c6 d2d4 c5d4 f3d4
info depth 19 seldepth 24 multipv 1 score cp -35 upperbound nodes 721201 nps 720480 hashfull 90 tbhits 0 time 1001 pv c7c5 g1f3
bestmove c7c5 ponder g1f3
> # engine move is c7c5, let's see the board
> board

+---+---+---+---+---+---+---+---+
| r | n | b | q | k | b | n | r | 8
+---+---+---+---+---+---+---+---+
| p | p | p | p | p | p | p | p | 7
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 6
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 5
+---+---+---+---+---+---+---+---+
|   |   |   |   | P |   |   |   | 4
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 3
+---+---+---+---+---+---+---+---+
| P | P | P | P |   | P | P | P | 2
+---+---+---+---+---+---+---+---+
| R | N | B | Q | K | B | N | R | 1
+---+---+---+---+---+---+---+---+
a   b   c   d   e   f   g   h

Fen: rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq - 0 1
Key: B46022469E3DD31B
Checkers:
> # engine move is not on the board, we need to send the position command
> position startpos moves e2e4 c7c5
> # let's see the board now
> board

+---+---+---+---+---+---+---+---+
| r | n | b | q | k | b | n | r | 8
+---+---+---+---+---+---+---+---+
| p | p |   | p | p | p | p | p | 7
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 6
+---+---+---+---+---+---+---+---+
|   |   | p |   |   |   |   |   | 5
+---+---+---+---+---+---+---+---+
|   |   |   |   | P |   |   |   | 4
+---+---+---+---+---+---+---+---+
|   |   |   |   |   |   |   |   | 3
+---+---+---+---+---+---+---+---+
| P | P | P | P |   | P | P | P | 2
+---+---+---+---+---+---+---+---+
| R | N | B | Q | K | B | N | R | 1
+---+---+---+---+---+---+---+---+
a   b   c   d   e   f   g   h

Fen: rnbqkbnr/pp1ppppp/8/2p5/4P3/8/PPPP1PPP/RNBQKBNR w KQkq - 0 2
Key: 4CA78BCE9C2980B0
Checkers:
> # ok board is updated now
> # lets quit
> quit