为什么 StandardOutput.Read() 永远不会返回？ （僵局？）

Question

Using C#, I want to automate a third-party Windows command-line program. Usually, it is an interactive console, you send commands, it may prompt for details, send back a result and display a prompt to ask for more commands. Typically:

c:\>console_access.exe
Prompt> version
2.03g.2321
Prompt> 

I used .NET classes Process and ProcessStartInfo along with redirections of stdin/stdout/stderr.

    public ConsoleAccess()
    {
        if (!File.Exists(consoleAccessPath)) throw new FileNotFoundException(consoleAccessPath + " not found");

        myProcess = new Process();
        ProcessStartInfo myProcessStartInfo = new ProcessStartInfo(consoleAccessPath, ""); // even "2>&1" as argument does not work; my code still hangs
        myProcessStartInfo.CreateNoWindow = true; 
        myProcessStartInfo.UseShellExecute = false; 
        myProcessStartInfo.RedirectStandardOutput = true;
        myProcessStartInfo.RedirectStandardError = true;
        myProcessStartInfo.RedirectStandardInput = true;
        //myProcessStartInfo.ErrorDialog = true; // I tried, to no avail.
        myProcess.StartInfo = myProcessStartInfo;

        outputQueue = new ConcurrentQueue<string>(); // thread-safe queue
        errorQueue = new ConcurrentQueue<string>();

        myProcess.Start();
        myStandardOutput = myProcess.StandardOutput;
        myStandardError = myProcess.StandardError;
        myStandardInput = myProcess.StandardInput;

        stdOutPumper = new Thread(new ThreadStart(PumpStdOutLoop));
        stdOutPumper.Start();
        stdErrPumper = new Thread(new ThreadStart(PumpStdErrLoop));
        stdErrPumper.Start();

        string empty = getResponse(); // check for prompt
        string version = getVersion(); // one simple command
    }
    // [...]
    private void PumpStdErrLoop()
    {
        while (true)
        {
            string message = myStandardError.ReadLine();
            errorQueue.Enqueue(message);
        }
    }

    private void PumpStdOutLoop()
    {
        while (true)
        {
            bool done = false;
            string buffer = "";
            //int blocksize = 1024;
            string prompt = "Prompt> ";
            while (!done)
            {
                //char[] intermediaire = new char[blocksize];
                //int res = myStandardOutput.Read(intermediaire, 0, blocksize);
                //buffer += new string(intermediaire).Substring(0, res);
                byte b = (byte)myStandardOutput.Read(); // I go byte per byte, just in case the char[] above is the source of the problem. To no avail.
                buffer += (char)b;
                done = buffer.EndsWith(prompt);
            }
            buffer = buffer.Substring(0, buffer.Length - prompt.Length);
            outputQueue.Enqueue(buffer);
        }
    }

Since this program returns "Prompt> " (important : without "\\n" at the end) when it's waiting for commands, I can't use myProcess.BeginOutputReadLine();

However, I have to use threads because I must listen stdout AND stderr at the same time.

This is why I used threads and thread-safe queues for a class producer/consumer pattern.

"You can use asynchronous read operations to avoid these dependencies and their deadlock potential. Alternately, you can avoid the deadlock condition by creating two threads and reading the output of each stream on a separate thread." source: http://msdn.microsoft.com/en-us/library/system.diagnostics.process.standardoutput%28v=vs.100%29.aspx

With this design, all sequences like * cmd -> result with no err (something on stdout, nothing on stderr) * cmd -> error (something on stderr, nothing on stdout) works as expected. no problem.

• cmd -> result with warning (something on both stderr and stdout) should work (I'm trying to reproduce this scenario)

however, for one command in particular -- a command that prompts for a password during its execution -- does not work:

• main thread principal loops forever on if (errorQueue.Count == 0 && outputQueue.Count == 0) { System.Threading.Thread.Sleep(500); }if (errorQueue.Count == 0 && outputQueue.Count == 0) { System.Threading.Thread.Sleep(500); }
• thread pumping stdout waits forever on byte b = (byte)myStandardOutput.Read();
• thread pumping stdout waits a line forever on string message = myStandardError.ReadLine();

What I don't get is why byte b = (byte)myStandardOutput.Read(); does not pump the message "password:". Nothing happens. I never get the first 'p'.

I feel I hit a deadlock scenario, but I do not understand why.

What's wrong?

(I don't think it is very relevant but I tried the above on .NET 4.0 with MS Visual Studio 2010 on Windows 7 32-bit.)

Answer 1

This is a very common failure mode for these kind of interactive console mode programs. The C runtime library automatically switches the stderr and stdout streams to buffered mode when it detects that output is being redirected. Important to improve throughput. So output goes into that buffer instead of getting directly written to the console. Getting your program to see the output requires the buffer to be flushed.

There are three scenarios where the buffer gets flushed. A flush occurs when the buffer is full, typically around 2 kilobytes. Or when the program writes a line terminator (\\n). Or when the program explicitly calls fflush(). The first two scenarios do not occur, not enough output and the program isn't using \\n. Which points at the problem, the original programmer forgot to call fflush(). Forgetting this is very common, the programmer simply never intended the program to be used other than in an interactive way.

Nothing can do about it, you'll need to ask the owner or author of the program to add fflush(). Maybe you can limp along by just assuming that the prompt is being written.