how to do boost::asio::spawn with io_service-per-CPU?

Question

My server is based on boost spawn echo server .

The server runs fine on single-core machine, not even one crash for several months. Even when it takes 100% CPU it still works fine.

But I need to handle more client requests, now I use multi-core machine. To use all the CPUs I run io_service on several thread, like this:

#include <boost/asio/io_service.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/spawn.hpp>
#include <boost/asio/steady_timer.hpp>
#include <boost/asio/write.hpp>
#include <boost/thread/thread.hpp>
#include <iostream>
#include <memory>
#include <thread>
using namespace std;

using boost::asio::ip::tcp;

class session : public std::enable_shared_from_this<session>{
public:
    explicit session(tcp::socket socket)
        : socket_(std::move(socket)),
        timer_(socket_.get_io_service()),
        strand_(socket_.get_io_service())
    {}

    void go()
    {
        auto self(shared_from_this());
        boost::asio::spawn(strand_, [this, self](boost::asio::yield_context yield)
        {
           try {
               char data[1024] = {'3'};
               for( ; ;) {
                   timer_.expires_from_now(std::chrono::seconds(10));
                   std::size_t n = socket_.async_read_some(boost::asio::buffer(data, sizeof(data)), yield);
                   // do something with data
                   // write back something
                   boost::asio::async_write(socket_, boost::asio::buffer(data, sizeof(data)), yield);
               }
           } catch(...)   {
               socket_.close();
               timer_.cancel();
           }
        });

        boost::asio::spawn(strand_, [this, self](boost::asio::yield_context yield)
        {
           while(socket_.is_open()) {
               boost::system::error_code ignored_ec;
               timer_.async_wait(yield[ignored_ec]);
               if(timer_.expires_from_now() <= std::chrono::seconds(0))
                   socket_.close();
           }
        });
    }

private:
    tcp::socket socket_;
    boost::asio::steady_timer timer_;
    boost::asio::io_service::strand strand_;
};

int main(int argc, char* argv[]) {
    try {

        boost::asio::io_service io_service;

        boost::asio::spawn(io_service, [&](boost::asio::yield_context yield)
        {
            tcp::acceptor acceptor(io_service,
#define PORT "7788"
            tcp::endpoint(tcp::v4(), std::atoi(PORT)));

            for( ; ;) {
                boost::system::error_code ec;
                tcp::socket socket(io_service);
                acceptor.async_accept(socket, yield[ec]);
                if(!ec)
                    // std::make_shared<session>(std::move(socket))->go();
                    io_service.post(boost::bind(&session::go, std::make_shared<session>(std::move(socket))));
            }
        });

        // ----------- this works fine on single-core machine ------------
        {
            // io_service.run();
        }

        // ----------- this crashes (with multi core) ----------
        {
            auto thread_count = std::thread::hardware_concurrency(); // for multi core
            boost::thread_group threads;
            for(auto i = 0; i < thread_count; ++i)
                threads.create_thread(boost::bind(&boost::asio::io_service::run, &io_service));

            threads.join_all();
        }

    } catch(std::exception& e) {
        std::cerr << "Exception: " << e.what() << "\n";
    }

    return 0;
}

The code works fine on single-core maching, but crashes all the time on 2-core/4-core/8-core machine. From the crash dump I don't see anything related to my code, just about something with boost::spawn and some randomly named lambda.

So I want to try this: Run io_service per CPU.

I found some demo , but it uses async function:

void server::start_accept()
{
  new_connection_.reset(new connection(
        io_service_pool_.get_io_service(), request_handler_));
  acceptor_.async_accept(new_connection_->socket(),
      boost::bind(&server::handle_accept, this,
        boost::asio::placeholders::error));
}

void server::handle_accept(const boost::system::error_code& e)
{
  if (!e)
  {
    new_connection_->start();
  }

  start_accept();
}

The io_service_pool_.get_io_service() randomly pickup an io_service , but my code uses spawn

boost::asio::spawn(io_service, ...

How to spawn with random io_service ?

Answer 1

Seems I was asking the wrong question, spawn cannot work with multiple io_service , but the socket can. I modified the code to this:

int main(int argc, char* argv[]) {
    try {

        boost::asio::io_service io_service;
        boost::asio::io_service::work work(io_service);

        auto core_count = std::thread::hardware_concurrency();
        // io_service_pool.hpp and io_service_pool.cpp from boost's example
        io_service_pool pool(core_count);

        boost::asio::spawn(io_service, [&](boost::asio::yield_context yield)
        {
#define PORT "7788"
            tcp::acceptor acceptor(io_service, tcp::endpoint(tcp::v4(), std::atoi(PORT)));

            for( ; ;) {
                boost::system::error_code ec;
                boost::asio::io_service& ios = pool.get_io_service();
                tcp::socket socket(ios);
                acceptor.async_accept(socket, yield[ec]);
                if(!ec)
                    ios.post(boost::bind(&session::go, std::make_shared<session>(std::move(socket))));
            }
        });

        { // run all io_service
            thread t([&] { pool.run(); });
            t.detach();

            io_service.run();
        }

    } catch(std::exception& e) {
        std::cerr << "Exception: " << e.what() << "\n";
    }

    return 0;
}

Now the server doesn't crash anymore. But I still have no idea what could cause the crash if I use a single io_service for all threads.