Boost ASIO/Coroutines: Attempting to write an echo server using boost asio and coroutines, but am getting inconsistent behaviour

Question

It appears that I misunderstood how windows handles sockets in TIME_WAIT when there are many sockets being opened. If too many are hanging out in TIME_WAIT, it just errors. Linux cleans up the older connections and succeeds (at least on my box, not sure where this is documented).

I am attempting to write an coroutine based echo server, but it appears to behave slightly randomly. I clearly am missing something. Can anyone tell me if my approach is wrong, and if so what I am missing?

Update: Tested this on linux (Ubuntu 14.04/gcc 4.8/boost 1.56), and everything seems fine. On windows the client starts throwing exceptions ( Client Exception: Only one usage of each socket address (protocol/network address/port) is normally permitted ) sometimes. Other times it runs fine.

Update Two: I assumed that something was screwed up in my code, but it appears that it is more an issue with my understanding of windows networking. It appears that when the socket is in TIME_WAIT on windows it sticks around even if new sockets are requested above some limit, on linux they are discarded when a new socket is requested if there are too many hanging around.

#include <iostream>
#include <thread>
#include <memory>
#include <atomic>

#include <boost/asio.hpp>
#include <boost/asio/spawn.hpp>

using namespace boost::asio::ip;
using namespace boost::asio;
using std::cout;
using std::endl;

std::atomic<int> active;

void echo_server(boost::asio::io_service &svc, int c)
{
    spawn(svc, [&svc, c](yield_context y) {
        try
        {
            tcp::acceptor acceptor(svc, tcp::endpoint(tcp::v4(), 6789));
            for (int i = 0; i < c; i++)
            {
                std::shared_ptr<tcp::socket> s = std::make_shared<tcp::socket>(svc);
                acceptor.async_accept(*s, y);

                spawn(y, [s](yield_context y) mutable {
                    try
                    {
                        streambuf buf;
                        for (;;)
                        {
                            async_read(*s, buf, transfer_at_least(1), y);
                            async_write(*s, buf, y);
                        }
                    }
                    catch (boost::system::system_error &e)
                    {
                        if (e.code() != error::eof)
                            cout << e.what() << endl;
                    }
                });
            }
            cout << "Server Done\n";
        }
        catch (std::exception &e)
        {
            std::cerr << "Server Exception: " << e.what() << std::endl;
        }
    });
}

void echo_client(boost::asio::io_service &svc)
{
    spawn(svc, [&svc](yield_context y) {
        tcp::socket sock(svc);
        try
        {
            async_connect(sock, tcp::resolver(svc).async_resolve(
                tcp::resolver::query("localhost", "6789"), y), y);

            char data[128];
            memcpy(data, "test", 5);
            async_write(sock, buffer(data), y);
            memset(data, 0, sizeof(data));
            async_read(sock, buffer(data), transfer_at_least(1), y);
            if (strcmp("test", data))
                cout << "Error, server is broken\n";

        }
        catch (std::exception &e)
        {
            std::cerr << "Client Exception: " << e.what() << std::endl;
        }

        active--;
    });
}

int main(int argc, char **argv)
{
    io_service svc;
    int c = 10000;
    active = 0;

    // Schedule a server
    echo_server(svc, c);

    // Schedule a bunch of clients, run only 1000 at a time.
    while (c > 0)
    {
        cout << "Remain " << c << endl;
        for (int i = 0; i < 1000; i++)
        {
            echo_client(svc);
        }
        active = 1000;
        while (active > 0)
        {
            if (svc.run_one() == 0) 
            {
                cout << "IO Service reset\n";
                svc.reset();
            }
        }
        c -= 1000;
    }
    svc.run();

    return 0;
}

Answer 1

I think there's some unexplained things in your driver. In particular, what are you resetting the service for? Try:

int main()
{
    boost::asio::io_service svc;
    int c = 100;

    // Schedule a server
    echo_server(svc, c);

    total_clients = 0;

    // Schedule a bunch of clients
    for (int i = 0; i < c; i++)
    {
        echo_client(svc);
    }

    std::cout << "total_clients: " << total_clients << "\n";
    svc.run();
}

Works for me:

total_clients: 100
Done handling new connections