Creating a separate boost thread for endpoint.listen() in a multithreaded program using websocketpp library

Question

I am trying to integrate a websocketpp server into a multithreaded project. Everything works fine in a single thread approach, but I encountered a problem when creating a separate boost::thread for endpoint.listen() that would run in the background (so it does not disrupt the execution of the main thread). I have tried the code with Boost v1.46.1 and v1.50.0 on Ubuntu 12.04 64-bit with the newest build of websocketpp. Below is a code sample and an explanation of my approach.

#include <websocketpp/websocketpp.hpp>
#include <boost/thread.hpp>
#include <boost/bind.hpp>
#include <exception>

using websocketpp::server;

class echo_server_handler : public server::handler {
public:
    void on_message(connection_ptr con, message_ptr msg) {
        con->send(msg->get_payload(),msg->get_opcode());
    std::cout << "Got message: " << msg->get_payload() << std::endl;
    }
};


int main(int argc, char* argv[]) {

    unsigned short port = 9002;

    try {       
        server::handler::ptr h(new echo_server_handler());
        server echo_endpoint(h);

        echo_endpoint.alog().unset_level(websocketpp::log::alevel::ALL);
        echo_endpoint.elog().unset_level(websocketpp::log::elevel::ALL);

        echo_endpoint.alog().set_level(websocketpp::log::alevel::CONNECT);
        echo_endpoint.alog().set_level(websocketpp::log::alevel::DISCONNECT);

        echo_endpoint.elog().set_level(websocketpp::log::elevel::RERROR);
        echo_endpoint.elog().set_level(websocketpp::log::elevel::FATAL);

        std::cout << "Starting WebSocket echo server on port " << port << std::endl;

        //Getting pointer to the right function
         void(websocketpp::role::server<websocketpp::server>::*f)(uint16_t,size_t) =
                &websocketpp::role::server<websocketpp::server>::listen;

        std::cout << "Starting WSServer thread... \n" << std:endl;
        boost::shared_ptr<boost::thread> ptr(new boost::thread(boost::bind(f, &echo_endpoint, port, 1)));
    //ptr->join();

    } catch (std::exception& e) {
        std::cerr << "Exception: " << e.what() << std::endl;
    }

    //Simulating processing in the main thread
    while(true) {std::cout << "Main thread processing..."<<std::endl; sleep(5);}


    return 0;

}

If I compile the code with ptr->join(); the listening thread works fine, but it makes the main thread sleep. If I leave ptr->join() out and let the listening thread run in background, I encounter an error after the thread creation:

/usr/local/boost_1_50_0/libbin/include/boost/thread/pthread/recursive_mutex.hpp:105: void boost::recursive_mutex::lock(): Assertion `!pthread_mutex_lock(&m)' failed.

I'm not very experienced with threading or boost threads, and quite new with websocketpp, so I'm not sure if I'm doing something wrong here. If there are any better (and working) ways to tackle this issue, I would love to see some examples. I have been trying to figure out this problem for a long time now, so any help would be priceless. Thanks in advance!

Check out also: gdb stacktrace and valgrind results

Edit: The "while(true)" in the code sample is there just to simulate the processing in the main thread. I'm integrating a websocket server in a big project that has different types of socket connections, events, data processing, client synchronization etc. running in the background. The websocket connection provides just another way to connect to the server using a web client instead a native one. The main thread creates all the necessary stuff, but I can't really affect in which order they are created, so the websocket server must be started in its own thread.

Answer 1

You create all the objects within the scope of try/catch. When you leave this scope, these objects get destroyed.

So, either move the object definitions out of try/catch, or move while(true) loop into it.

Answer 2

Why are you creating the boost::thread on the heap, when it could be on the stack?

You don't need to use boost::bind with boost::thread , so it should be simply:

    boost::thread t(f, &echo_endpoint, port, 1);

Much simpler, no?

As for your program's behaviour. If you call ptr->join() there then the main thread waits for the other thread to finish, which never happens, so of course it sleeps. If you don't join it then ptr and echo_endpoint and h all go out of scope. The other thread will then be trying to use objects which no longer exist.

As @IgorR. said , you should put the while loop inside the try-catch block, so the work in the main loop can happen before the other thread and the objects it uses go out of scope.

From Boost 1.50 the boost::thread destructor matches the behaviour of std::thread ie it calls terminate() if the thread is joinable when its destructor runs. This is to prevent the sort of error you have, where the thread continues running even though the boost::thread handle referring to it and other stack objects no longer exist. If you want it to keep running you must detach it explicitly (but in your program that would still be wrong, because the echo_endpoint and h objects would still cease to exist and the thread would still try to use them.) So before a boost::thread object goes out of scope you should either join it or detach it.