boost :: asio http客户端停止工作，我不知道为什么

Question

I have a program that uses boost::asio to assign work to several threads. The work consists of starting a http client, making a request and storing the answer in a file. Sometimes there is a bug which causes the program to never finish and stop writing any output. I've been unable to figure out what exactly is going wrong, as the program doesn't report any errors or problems that could explain this behavior (it does report the occasional timeout or some other minor issue).

I'm on Windows and typing "netstat -n" into the console shows that the program maintains 8 established connections with the target host even long after it has stopped doing work (one connection for each thread).

Mutexes that are used:

std::mutex catch_mx, result_mx, debug_mx;

Assigning work to threads:

    boost::asio::io_service io_service;
    for (auto &wordset : wordsets)
    for (auto &unicode_string : wordset.variants)
        io_service.post(std::bind(send_query, std::ref(io_service), std::ref(unicode_string)));

    std::vector<std::thread> threads;
    threads.reserve(std::max(1u, std::thread::hardware_concurrency()));
    for (auto i = 0u; i < threads.capacity(); ++i)
        threads.emplace_back(thread_function, std::ref(io_service));

    for (auto &t : threads)
        t.join();

Allow the thread to receive work:

void thread_function(boost::asio::io_service &io_service)
{
    io_service.run();
}

The function that makes the http request and interprets the response. The http client code has been copied from the boost::asio synchronous http client example. The only difference is in the error handling and the response being written to a file rather than std::cout

void send_query(boost::asio::io_service &io_service, const Ustring &unicode_string)
{
    try
    {
        using boost::asio::ip::tcp;
        auto query_string = generate_query(unicode_string);
        debug_log(unicode_string, query_string);
        tcp::resolver resolver(io_service);
        tcp::resolver::query query("somehost.com", "http");
        tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
        tcp::socket socket(io_service);
        boost::asio::connect(socket, endpoint_iterator);
        boost::asio::streambuf request;
        std::ostream request_stream(&request);
        request_stream << "GET " << "somepath" + query_string << " HTTP/1.0\r\n";
        request_stream << "Host: " << "somehost.com" << "\r\n";
        request_stream << "Accept: */*\r\n";
        request_stream << "Connection: close\r\n\r\n";
        boost::asio::write(socket, request);
        boost::asio::streambuf response;
        boost::asio::read_until(socket, response, "\r\n");
        std::istream response_stream(&response);
        std::string http_version;
        response_stream >> http_version;
        unsigned int status_code;
        response_stream >> status_code;
        std::string status_message;
        std::getline(response_stream, status_message);
        if (!response_stream || http_version.substr(0, 5) != "HTTP/")
            throw AUTO_EXCEPTION("invalid response");
        if (status_code != 200) // for now, consider this an error
            throw AUTO_EXCEPTION("response status code " + std::to_string(status_code));

        boost::asio::read_until(socket, response, "\r\n\r\n");
        std::stringstream ss;
        std::string header;
        while (std::getline(response_stream, header) && header != "\r");
        ss << header << "\n";
        ss << "\n";

        if (response.size() > 0)
            ss << &response;

        boost::system::error_code error;
        while (boost::asio::read(socket, response, boost::asio::transfer_at_least(1), error))
            ss << &response;
        if (error != boost::asio::error::eof)
            throw AUTO_EXCEPTION(error.message());

        write_result(ss.str());
    }
    catch (const std::exception &e)
    {

        std::unique_lock<std::mutex> lock(catch_mx);
        std::ofstream ofs("error.log", std::ios_base::app);
        ofs << "Thread " << std::this_thread::get_id() << ": " << e.what() << std::endl;
        ofs.close();
    }
}

Logging functions

void debug_log(const Ustring &code_points, std::string &query)
{
    std::unique_lock<std::mutex> lock(debug_mx);
    std::ofstream ofs("debug.log", std::ios_base::app);
    ofs << unicode_to_string(code_points) << " " << query << std::endl;
    ofs.close();
}

void write_result(const std::string &s)
{
    std::unique_lock<std::mutex> lock(result_mx);
    std::ofstream ofs("results.txt", std::ios_base::app);
    ofs << s << std::endl;
    ofs.close();
}

PS: doing as AndyT suggested, I found that the threads all seem to get stuck at the same step in one of boost::asio's functions (in socket_ops.ipp):

signed_size_type recv(socket_type s, buf* bufs, size_t count,
    int flags, boost::system::error_code& ec)
{
  clear_last_error();
#if defined(BOOST_ASIO_WINDOWS) || defined(__CYGWIN__)
  // Receive some data.
  DWORD recv_buf_count = static_cast<DWORD>(count);
  DWORD bytes_transferred = 0;
  DWORD recv_flags = flags;
  int result = error_wrapper(::WSARecv(s, bufs,
        recv_buf_count, &bytes_transferred, &recv_flags, 0, 0), ec); // this is where they all get stuck
  if (ec.value() == ERROR_NETNAME_DELETED)
    ec = boost::asio::error::connection_reset;
  else if (ec.value() == ERROR_PORT_UNREACHABLE)
    ec = boost::asio::error::connection_refused;
  if (result != 0)
    return socket_error_retval;
  ec = boost::system::error_code();
  return bytes_transferred;
#else // defined(BOOST_ASIO_WINDOWS) || defined(__CYGWIN__)
  msghdr msg = msghdr();
  msg.msg_iov = bufs;
  msg.msg_iovlen = static_cast<int>(count);
  signed_size_type result = error_wrapper(::recvmsg(s, &msg, flags), ec);
  if (result >= 0)
    ec = boost::system::error_code();
  return result;
#endif // defined(BOOST_ASIO_WINDOWS) || defined(__CYGWIN__)
}

Answer 1

This looks like deadlock. It is common problem for asynchronous code that use locks. Try to use strands instead of locks in your boost::asio code. You can post handlers to your io_service and wrap them with different strands. One strand for debug, one for writing output and one for handling errors. For example, when you need to write debug information - you need to create handler that do it, than wrap it with corresponding strand and than - post it to io_service.

It is better to use asynchronous operations for all I/O.