How to get output from a long-running child process in C++

Question

From a C++ program, I am wanting to:

• Launch a child process.
• Wait until it emits a line of output.
• Capture that line of output, and allow the child process to continue running.

This feels like it should be trivial, but I've tried it now two ways, and have hit a similar roadblock each time. First I used boost::process , like so:

auto exe = boost::process::search_path("other_executable");
boost::process::ipstream output;
master_process_ = boost::process::child(exe, boost::process::std_out > output);
std::string important_line;
std::getline(output, important_line);

And the second approach was to just use popen :

FILE* master_process_ = popen("other_executable", "r");
char stdout_buffer[128];
while (fgets(stdout_buffer, sizeof(stdout_buffer), master_process_)) {
   // log print here
}

In both cases, the program blocks when it tries to read from the pipe— in getline and fgets , respectively, with gdb showing it stuck in the low-level read function:

#0  0x00007ffff7dcc332 in __libc_read (fd=3, buf=0x555555581d40, nbytes=895) at ../sysdeps/unix/sysv/linux/read.c:26
#1  0x00007ffff794052c in read (__nbytes=<optimized out>, __buf=<optimized out>, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
#2  boost::process::detail::posix::basic_pipe<char, std::char_traits<char> >::read (count=<optimized out>, data=<optimized out>, this=0x7fffffffcbd0)
    at /usr/include/boost/process/detail/posix/basic_pipe.hpp:93
#3  boost::process::basic_pipebuf<char, std::char_traits<char> >::underflow (this=0x7fffffffcb90) at /usr/include/boost/process/pipe.hpp:202
#4  0x00007ffff7c2351a in std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, char) () from /lib/x86_64-linux-gnu/libstdc++.so.6

For further context, other_executable is a small Python program which acts as expected when run on its own (emits output, sticks around). It also opens a server, and I can communicate with it fine in both of these scenarios, so it's definitely getting run (GDB's detach message further confirms this, as does ps ).

What am I doing wrong here?

Answer 1

Turns out both implementations were working fine— the issue was that the Python script wasn't flushing stdout. I thought I was in the clear here because I was emitting a \\n and it seemed to be fine when invoked from the terminal, but clearly not.

Thanks for the pointer on this, @paul-sanders.