How to recover from ENFILE in acceptor loop

Question

I have written a test to show a problem with asio async_accept I have a server that leave any connection to it open forever after accepting many connections (in my case 1017) the next connect fails with "Too many open files" error. and then any call to async_accept invokes the handler imediately

Is this a misunderstanding?

I use debian 7 amd64

Makefile

CXX=clang++ -O2
OBJ= main.o server.o
LIBS=-lboost_system -lboost_thread
all: server

server: $(OBJ)
    $(CXX) -o server $(OBJ) $(LIBS)

main.o: main.cpp
    $(CXX) -c main.cpp

server.o: server.hpp server.cpp
    $(CXX) -c server.cpp

clean:
    rm -f *.o *~

distclean: clean
    rm -f server

server.hpp

#include <boost/asio.hpp>

class server
{
    public:
        server();

        void run();

    private:
        void start_accept();

        void handle_accept(const boost::system::error_code& e);

        boost::asio::io_service ios;

        boost::asio::ip::tcp::acceptor acceptor_;

        std::vector<boost::asio::ip::tcp::socket*> sockets;
};

server.cpp

#include <boost/bind.hpp>
#include "server.hpp"

server::server(): acceptor_(ios)
{
    boost::asio::ip::tcp::resolver resolver(ios);
    boost::asio::ip::tcp::resolver::query query("0.0.0.0", "5050");
    boost::asio::ip::tcp::endpoint endpoint = *resolver.resolve(query);
    acceptor_.open(endpoint.protocol());
    acceptor_.set_option(boost::asio::ip::tcp::acceptor::reuse_address(true));
    acceptor_.bind(endpoint);
    acceptor_.listen();

    start_accept();
}

void server::run()
{
    ios.run();
}

void server::start_accept()
{
    boost::asio::ip::tcp::socket *s=new  boost::asio::ip::tcp::socket(ios);
    sockets.push_back(s);
    acceptor_.async_accept(*s,
            boost::bind(&server::handle_accept, this,
                boost::asio::placeholders::error));
}

void server::handle_accept(const boost::system::error_code& e)
{
    if(e)
    {
        std::cerr<<"e.message() = "<<e.message()<<std::endl;
        boost::asio::ip::tcp::socket *s=sockets.back();
        s->close();
        delete s;
        sockets.pop_back();
    }
    static int i=1;
    std::cerr<<"i = "<<i++<<std::endl;
    start_accept();
}

main.cpp

#include "server.hpp"

int main()
{
    server s;
    s.run();
    return 0;
}

and my test is

for x in `seq $1`;do nc 127.0.0.1 5050 & done

Answer 1

At the end of server::handle_accept you submit the asynchronous accept back onto the IO service queue, even though the socket is in error.

I'm not sure this is what you want, but I can make it "work" (recover) by starting the listener fresh on accept failure. (Note that this would require some synchronization if you run the service on multiple threads).

Live On Coliru

#include <boost/asio.hpp>

class server
{
    public:
        server();

        void run();

    private:
        bool start_listen();
        void start_accept();
        void handle_accept(boost::system::error_code e);

        boost::asio::io_service ios;
        boost::asio::ip::tcp::acceptor acceptor_;
        std::vector<boost::asio::ip::tcp::socket*> sockets;
};

#include <boost/bind.hpp>

server::server(): acceptor_(ios)
{
    start_listen();
    start_accept();
}

void server::run()
{
    ios.run();
}

bool server::start_listen()
{
    boost::system::error_code e;
    boost::asio::ip::tcp::endpoint endpoint { {}, 5050 };
    acceptor_.open(endpoint.protocol());
    acceptor_.set_option(boost::asio::ip::tcp::acceptor::reuse_address(true));
    acceptor_.bind(endpoint, e);
    if (e)
        return false;
    acceptor_.listen();
    return true;
}

void server::start_accept()
{
    boost::asio::ip::tcp::socket *s=new  boost::asio::ip::tcp::socket(ios);
    sockets.push_back(s);
    acceptor_.async_accept(*s, boost::bind(&server::handle_accept, this, boost::asio::placeholders::error));
}

#include <iostream>

void server::handle_accept(boost::system::error_code e)
{
    if(e)
    {
        std::cerr<<"e.message() = "<<e.message()<<std::endl;
        boost::asio::ip::tcp::socket *s=sockets.back();
        s->close();
        delete s;
        sockets.pop_back();

        acceptor_.close();
        if (!start_listen())
            return;
    }
    static int i=1;
    std::cerr<<"i = "<<i++<<std::endl;
    start_accept();
}

int main()
{
    server s;
    s.run();
}

Answer 2

When you open the socket this way

boost::asio::ip::tcp::socket *s=new  boost::asio::ip::tcp::socket(ios);

the socket is not opened, just an object is created. When you try to accept it, you hit the open file decriptor limit, so you get "too many open files" error. Then you just remove it from vector, and create another socket that will not be able to be opened in accept. I think that's the reason of an error.

Update: By the way, this function

void server::start_accept()
{
    boost::asio::ip::tcp::socket *s=new  boost::asio::ip::tcp::socket(ios);
    sockets.push_back(s);
    acceptor_.async_accept(*s,
        boost::bind(&server::handle_accept, this,
            boost::asio::placeholders::error));
}

seems to have a possible memory leak, if exception is generated in push_back method, s will be lost.

Update: the inifinte loop is fixed if you rewrite handle_accept this way:

void server::handle_accept(const boost::system::error_code& e)
{
    if(e)
    {
      auto size = 2;
      for (int i = 0; i < size; ++i) {
        std::cerr<<"e.message() = "<<e.message()<<std::endl;
        boost::asio::ip::tcp::socket *s=sockets.back();
        s->close();
        delete s;
        sockets.pop_back();
      }
    }
    static int i=1;
    std::cerr<<"i = "<<i++<<std::endl;
    start_accept();
}