Golang os/exec 刷新标准输入而不关闭它

Question

I would like to manage a process in Go with the package os/exec . I would like to start it and be able to read the output and write several times to the input.

The process I launch in the code below, menu.py , is just a python script that does an echo of what it has in input.

func ReadOutput(rc io.ReadCloser) (string, error) {
    x, err := ioutil.ReadAll(rc)
    s := string(x)
    return s, err
}

func main() {
    cmd := exec.Command("python", "menu.py")
    stdout, err := cmd.StdoutPipe()
    Check(err)

    stdin, err := cmd.StdinPipe()
    Check(err)

    err = cmd.Start()
    Check(err)

    go func() {
        defer stdin.Close() // If I don't close the stdin pipe, the python code will never take what I write in it
        io.WriteString(stdin, "blub")
    }()

    s, err := ReadOutput(stdout)
    if err != nil {
        Log("Process is finished ..")
    }
    Log(s)

    // STDIN IS CLOSED, I CAN'T RETRY !
}

And the simple code of menu.py :

while 1 == 1:
    name = raw_input("")
    print "Hello, %s. \n" % name

The Go code works, but if I don't close the stdin pipe after I write in it, the python code never take what is in it. It is okay if I want to send only one thing in the input on time, but what is I want to send something again few seconds later? Pipe is closed! How should I do? The question could be "How do I flush a pipe from WriteCloser interface?" I suppose

Answer 1

I think the primary problem here is that the python process doesn't work the way you might expect. Here's a bash script echo.sh that does the same thing:

#!/bin/bash

while read INPUT
  do echo "Hello, $INPUT."
done

Calling this script from a modified version of your code doesn't have the same issue with needing to close stdin :

func ReadOutput(output chan string, rc io.ReadCloser) {
    r := bufio.NewReader(rc)
    for {
        x, _ := r.ReadString('\n')
        output <- string(x)
    }
}

func main() {
    cmd := exec.Command("bash", "echo.sh")
    stdout, err := cmd.StdoutPipe()
    Check(err)

    stdin, err := cmd.StdinPipe()
    Check(err)

    err = cmd.Start()
    Check(err)

    go func() {
        io.WriteString(stdin, "blab\n")
        io.WriteString(stdin, "blob\n")
        io.WriteString(stdin, "booo\n")
    }()

    output := make(chan string)
    defer close(output)
    go ReadOutput(output, stdout)

    for o := range output {
        Log(o)
    }
}

The Go code needed a few minor changes - ReadOutput method needed to be modified in order to not block - ioutil.ReadAll would have waited for an EOF before returning.

Digging a little deeper, it looks like the real problem is the behaviour of raw_input - it doesn't flush stdout as expected. You can pass the -u flag to python to get the desired behaviour:

cmd := exec.Command("python", "-u", "menu.py")

or update your python code to use sys.stdin.readline() instead of raw_input() (see this related bug report: https://bugs.python.org/issue526382 ).

Answer 2

Even though there is some problem with your python script. The main problem is the golang pipe. A trick to solve this problem is use two pipes as following:

// parentprocess.go
package main

import (
    "bufio"
    "log"
    "io"
    "os/exec"
)

func request(r *bufio.Reader, w io.Writer, str string) string {
    w.Write([]byte(str))
    w.Write([]byte("\n"))
    str, err := r.ReadString('\n')
    if err != nil {
        panic(err)
    }
    return str[:len(str)-1]
}

func main() {
    cmd := exec.Command("bash", "menu.sh")
    inr, inw := io.Pipe()
    outr, outw := io.Pipe()
    cmd.Stdin = inr
    cmd.Stdout = outw

    if err := cmd.Start(); err != nil {
        panic(err)
    }
    go cmd.Wait()
    reader := bufio.NewReader(outr)
    log.Printf(request(reader, inw, "Tom"))
    log.Printf(request(reader, inw, "Rose"))
}

The subprocess code is the same logic as your python code as following:

#!/usr/bin/env bash
# menu.sh
while true; do
    read -r name
    echo "Hello, $name."
done

If you want to use your python code you should do some changes:

while 1 == 1:
    try:
        name = raw_input("")
        print "Hello, %s. \n" % name
        sys.stdout.flush() # there's a stdout buffer
    except:
        pass # make sure this process won't die when come across 'EOF'

Answer 3

// StdinPipe returns a pipe that will be connected to the command's
// standard input when the command starts.
// The pipe will be closed automatically after Wait sees the command exit.
// A caller need only call Close to force the pipe to close sooner.
// For example, if the command being run will not exit until standard input`enter code here`
// is closed, the caller must close the pipe.
func (c *Cmd) StdinPipe() (io.WriteCloser, error) {}