如何在进程运行时期间捕获标准输出并打印到控制台和文件（C ++ / Boost.Process）

Question

I am working on an application that starts child processes using Boost's C++ Process library ( http://www.highscore.de/boost/process0.5 ) and redirects the standard out of that process to a file using the below code: boost::process::initializers::bind_stdout(boost::iostreams::file_descriptor_sink goes here )

The above solution and code worked well.

However, now, I need to be able to print the child process's standard out to both a file and the console.

I have found the code below which seems that it should do the job:

#include <boost/process.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <iostream>

using namespace boost::process;
using namespace boost::process::initializers;
using namespace boost::iostreams;

int main()
{
    boost::process::pipe p = create_pipe();
    {
        file_descriptor_sink sink(p.sink, close_handle);
        execute(
            run_exe("/bin/echo"),
            set_cmd_line("echo hello"),
            bind_stdout(sink)
        );
    }
    file_descriptor_source source(p.source, close_handle);
    stream<file_descriptor_source> is(source);
    std::string s;
    while (std::getline(is, s))
    {
            std::cout << s << std::endl;
            //will write line by line to file here
    }
}

I have tried running the above code and indeed I was able to print the output to the console and a file. But there is one major issue with the above code (at least in my case), the while loop ( while (std::getline(is, s)) ) only starts after the invoked process has exited, which means that all the output that was accumulated during the process run is printed at once once the process finishes.

Now, What I need is to be able to get the output from the invoked process during it's runtime and print it to both the console and the file as it comes (rather than only at the end).

(Since the processes that I start are out of my control and can take even a few good minutes to run, and currently the screen console is just empty until the process exits).

I have went over a lot of websites trying to find a solution for this one, as well as playing a lot with the code, but I can't seem to nail the solution yet.

How can this be accomplished? (Is it even possible with Boost.Process?) How would I be able to make that string/stream populate with the std out of the child process during it's runtime rather that only when it's finished?

Answer 1

In principle it should work as you require.

Perhaps you need to flush. (Try using std::unitbuf ).

Alternatively, you might be on Windows, which doesn't support async pipes readily. The docs describe a create_async_pipe function:

bp::pipe create_async_pipe()
{
#if defined(BOOST_WINDOWS_API)
    std::string name = "\\\\.\\pipe\\boost_process_async_io";
    HANDLE handle1 = ::CreateNamedPipeA(name.c_str(), PIPE_ACCESS_INBOUND |
        FILE_FLAG_OVERLAPPED, 0, 1, 8192, 8192, 0, NULL);
    HANDLE handle2 = ::CreateFileA(name.c_str(), GENERIC_WRITE, 0, NULL,
        OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
    return make_pipe(handle1, handle2);
#elif defined(BOOST_POSIX_API)
    return create_pipe();
#endif
}

So, here's a version that combines it all, complete with tee device:

#include <boost/process.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/tee.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <iostream>
#include <fstream>

namespace bp = boost::process;
namespace io = boost::iostreams;
using namespace bp;
using namespace bp::initializers;

bp::pipe create_async_pipe()
{
#if defined(BOOST_WINDOWS_API)
    std::string name = "\\\\.\\pipe\\boost_process_async_io";
    HANDLE handle1 = ::CreateNamedPipeA(name.c_str(), PIPE_ACCESS_INBOUND |
        FILE_FLAG_OVERLAPPED, 0, 1, 8192, 8192, 0, NULL);
    HANDLE handle2 = ::CreateFileA(name.c_str(), GENERIC_WRITE, 0, NULL,
        OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
    return make_pipe(handle1, handle2);
#elif defined(BOOST_POSIX_API)
    return create_pipe();
#endif
}

int main()
{
    bp::pipe p = create_async_pipe();

    {
        io::file_descriptor_sink sink(p.sink, io::close_handle);
        execute(
            run_exe("/bin/sh"),
            set_cmd_line("sh -c \"echo started; sleep 3 && date\""),
            bind_stdout(sink)
        );
    }

    io::file_descriptor_source source(p.source, io::close_handle);
    io::stream<io::file_descriptor_source> is(source);

    std::cout << "Process is running, let's wait before printing\n";

    system("sleep 1");

    std::ofstream ofile("test.log");
    io::filtering_ostream filt(io::tee(std::cout << std::unitbuf, ofile << std::unitbuf));

    std::string s;
    while (std::getline(is, s))
        filt << s << "\n";
}

The commands have been chosen in such a way that you can verify that it works in async fashion.

If you want "true" asynchrony, you can also employ Boost Asio, see the documentation for details: http://www.highscore.de/boost/process0.5/boost_process/tutorial.html#boost_process.tutorial.asynchronous_i_o