在发出CTRL + C命令之前，Popen读取数据

Question

I have a command line tool written in C for windows that scans for bluetooth devices via a serial USB Dongle.

It will loop through all devices in range repeating until it receives a CTRL+C command.

device1 name firmware
device2 name firmware
device3 name firmware
device1 name firmware
device2 name firmware
device3 name firmware
...

I want to stop the scan when it reaches a certain device so I can issue an update firmware command.

At the moment I can only capture the output after the CTRL+C command is issued using the following function that starts the scan, sleeps, then issues the CTRL+C command, I then catch the error and process the output in the except block:

        command = [self.cli_tool, '-s']

        startupinfo = subprocess.STARTUPINFO()
        startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
        startupinfo.wShowWindow = subprocess.SW_HIDE

        stream = []

        if self.check_dongle_firmware() is not False:
            try:                    
                self.proc = subprocess.Popen(
                            command, 
                            stdin=subprocess.PIPE, 
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.DEVNULL, 
                            startupinfo=startupinfo)

                time.sleep(SCAN_TIMEOUT)
                os.kill(self.proc.pid, signal.CTRL_C_EVENT)
                self.proc.wait()

            except KeyboardInterrupt:                
                for line in self.proc.stdout:
                    stream.append(line)                         

                for x in stream[7:]:
                    x = x.decode()
                    print(x.strip())   

I want something like this:

stream = []

 self.proc = subprocess.Popen(
                            command, stdin=subprocess.PIPE,
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.DEVNULL, 
                            startupinfo=startupinfo)

for line in self.proc.stdout:
    stream.append(line) 
    if 'device2' in line.decode().split():
        os.kill(self.proc.pid, signal.CTRL_C_EVENT)
        self.proc.wait()

This does not work though. It will not read 1 line at a time and the CTRL+C event is never reached.

I need to read and process 1 line at a time so that i can stop the process at a certain device.

Currently it only reads an empty byte or string depending on the pipe attributes.

printing to the screen (With the startup info off) or saving to a file without issuing a CTRL+C will render empty. I tried various combinations using :

universal_newlines=True

bufsize=1

self.proc.communicate()[0]

When I try the code using a command like ping I have no problem and full control of the output.

I have searched SO for tried anything that looks similar but nothing works for what I need.

I think I am missing something obvious or it is not possible due to the tool not applying a flush command in the C code?

Any direction or advice is appreciated!

In the C code I have fflush only in the following locations which i presume to be for error handling.

project.h in both bootloader\\src\\common\\ and receiver\\src\\common\\

//*************************************************************************
//------------------------- assertion of errors  --------------------------
//*************************************************************************
/*
 *   The format is assert(eval, error);
 *
 *   If 'eval' equals 0 then there is an error printed with number 'error'.
 *
 */

#define ERR_STACKOVERFLOW 0   // Out of stack space
#define ERR_SCH_OVERFLOW  1   // Scheduler overflow
#define ERR_SCH_OUTRANGE  2   // Scheduler out of range
#define ERR_WSHRS_OUTRANGE  3   // Scheduler out of range

#ifdef __nDEBUG__

#include "./error/error.h"
....
#include <stdio.h>
#define DBG_PRINT(a, args...) { printf(a, ##args);fflush(stdout); }

Answer 1

The problem here is probably, that the output of your CLI program is block buffered, since the output file descriptor is not associated with a terminal. You claim, that it works with "ping" instead of your cli program, so it definitively has to be an issue with the subprocess itself.

Since you programmed the cli program yourself, make sure that it does not buffer the output (ie put an fflush() after each line which is printed.

Specifically, adjust the code looks like this:

while (somecondition) {
    ...
    printf("%s %s %s\n", device, name, firmware);
    fflush(stdout); // <- Add this line after the printf
    ...
}

Alternatively, as melpomene suggested, you can also switch stdout to be line buffered early in the program with:

setvbuf(stdout, NULL, _IOLBF, 0);

Online C11 standard , 7.21.3/3:

When a stream is unbuffered , characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block. When a stream is fully buffered , characters are intended to be transmitted to or from the host environment as a block when a buffer is filled. When a stream is line buffered , characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered. Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment. Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.

7.21.3/7:

At program startup, three text streams are predefined and need not be opened explicitly — standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.