pool of processes using fork() and boost::asio

Question

I am currently trying to implement a process-pool that can be communicated with by the parent process. No child processes should exit until the parent tells thems so (likely using a signal). Upon now a couple of questions arose in my head and I am happy to get some input using my MWE:

#include <iostream>

#include <boost/thread.hpp>
#include <boost/process/async_pipe.hpp>
#include <boost/asio.hpp>
#include <boost/array.hpp>

static const std::size_t process_count = 3;

static void start_reading(boost::process::async_pipe& ap) 
{
    static boost::array<char, 256> buf;

    ap.async_read_some(boost::asio::buffer(buf.data(), buf.size()), [](const boost::system::error_code& error, std::size_t bytes_transferred)
    {
        if(!error)
        {
            std::cout << "received " << bytes_transferred << " from pid " << getpid() << " " << buf[0] << "...." << std::endl;
            // perform some heavy computations here..
        }
    });
}

static void start_writing(boost::process::async_pipe& ap)
{
    boost::array<char, 256> buf;
    buf.fill('A');

    ap.async_write_some(boost::asio::buffer(buf.data(), buf.size()), [&ap](const boost::system::error_code& error, std::size_t bytes_transferred)
    {
        if(!error)
        {
            std::cout << "parent " << getpid() << " sent " << bytes_transferred << " to [" << ap.native_source() << "," << ap.native_sink() << "]" << std::endl;
        }
    });
}

int main()
{
    try
    {        
        boost::asio::io_service io_context;

        // prevent the associated executor from stopping
        boost::asio::executor_work_guard<boost::asio::io_context::executor_type> guard = boost::asio::make_work_guard(io_context);

        pid_t main_process = getpid();
        std::cout << "before forks " << main_process << std::endl;

        std::vector<boost::process::async_pipe> pipes;
        pipes.reserve(process_count);

        for(std::size_t i = 0; i < process_count; i++)
        {
            pipes.emplace_back(io_context);
         
            io_context.notify_fork(boost::asio::io_service::fork_prepare);
            pid_t pid = fork();

            if(pid == 0)
            {
                io_context.notify_fork(boost::asio::io_service::fork_child);
                
                // perform some costly initialization here...

                boost::process::async_pipe& ap = pipes[i];
                
                std::cout << "child " << getpid() << " listening to [" << ap.native_source() << "," << ap.native_sink() << "]" << std::endl;
         
                start_reading(ap);

                io_context.run();               
            }
            else if(pid > 0)
            {
                io_context.notify_fork(boost::asio::io_service::fork_parent);
            }
            else
            {
                std::cerr << "fork() failed" << std::endl;
            }            
        }

        // only parent gets there
        start_writing(pipes[0]);
        start_writing(pipes[0]);
        start_writing(pipes[1]);
        start_writing(pipes[2]);

        io_context.run();
        
    }
    catch(const std::exception& e)
    {
        std::cerr << e.what() << std::endl;
    }
    
    return 1;
}

The program outputs

before forks 15603
child 15611 listening to [8,9]
child 15612 listening to [10,11]
parent 15603 sent 256 to [8,9]
parent 15603 sent 256 to [8,9]
parent 15603 sent 256 to [10,11]
parent 15603 sent 256 to [21,22]
received 256 from pid 15612 A....
received 256 from pid 15611 A....
child 15613 listening to [21,22]
received 256 from pid 15613 A....

My main concern at the time is how to infinitely read data in the worker processes (the childs) as long as the process is not already busy. As soon as the worker gets into the handler from async_read_some, it performs some computations as stated in the comment (might take a few seconds). While doing this, the process should and will block, afterwards I want to notify my parent to be ready again and accept new reads over the pipe. So far I don't have any profound idea how to do this. Notifying the parent from the child is not necessary per-se, but the parent needs to keep track of all idle child processes all time, so it can send new input via the corresponding pipe.

Apart from that there is one thing I didn't get yet: Notice that boost::array<char, 256> buf; is static in start_reading . If I remove the static modifier I never get into the completion handler of async_read_some , why is that?

EDIT: calling start_reading again in completion routine, will continue read. However, without the parent process "knowing" it.

EDIT2: Until now I figured out one possible way (guess there are several) that might work. I am not finished with the implementation but the shared mutex works as expected. Here is some pseudo-code:

process_pool
{

    worker get_next_worker()
        ScopedLock(memory_mapped_mutex);
        free_worker = *available.rbegin()
        available.pop_back()
        return free_worker;

    memory_mapped_vec<worker> available;

};

server::completion_handler_async_connect()
    get_next_worker().socket().write(request)


worker::completion_handler_async_read()
   // do something very long before locking
   ScopedLock(memory_mapped_mutex);
   process_pool::available.push_back(self);

Answer 1

Apart from that there is one thing I didn't get yet: Notice that boost::array<char, 256> buf; is static in start_reading. If I remove the static modifier I never get into the completion handler of async_read_some, why is that?

That's because the buf is a local varuable, and it doesn't exist after start_reading exits. However async_read (or any other async_XXXX call) returns immediately, without waiting for the operation to complete. So if the buffer doesn't persist then you are writing into aa dangling reference to unspecified stack space, leading to Undefined Behaviour .

As for communicating back and forth, that is unnecessarily complicated between processes. Is there any reason you can't use multi-threading? That way all workers can simply monitor a shared queue.

Of course you can setup the same with a queue shared between processes (in which case I would advise against doing it via pipes with Asio, but instead use message_queue .