Boost asio async_read_until followed by async_read

Question

I used the async tcp server example from boost which is near to what my application is doing. The code example below is a fully working example.

At first I start an async read operation until the delimiter char. In this case it is the http header complete sequence. The request contains some payload which is "hello world" (11 bytes). For a simplified example I use lambda handlers here. The first handler is called wit a length of 148 which is the header including four bytes for the delimiter sequence.

The size of the buffer gives me 159 which is the whole request including the payload. So far everything works as expected. To receive the payload I call another async read operation but the handler is never called. The first I tried was to read 11 bytes but that didn't work so I tried to read just two bytes to check if it's working buts it isn't.

The streambuf already contains all the data but why is the async read handler not called. Shouldn't it be called immediately because the data is inside the buffer or is there any misuse of the api?

Edit1:

I ended up checking calculating the bytes to read with the bytes inside the buffer. When there is no need for a "true" read operation I use io_server::post to add a wrapper for the handler. This seems as the best option for that purpose.

#include <cstdlib>
#include <iostream>
#include <memory>
#include <utility>
#include <boost/asio.hpp>

using boost::asio::ip::tcp;

class session
  : public std::enable_shared_from_this<session>
{
public:
  session(tcp::socket socket)
    : socket_(std::move(socket))
  {
  }

    boost::asio::streambuf buf;

  void start()
  {
    do_read();
  }

private:
  void do_read()
  {
    auto self(shared_from_this());

    boost::asio::async_read_until(socket_,buf, "\r\n\r\n", [this, self](boost::system::error_code ec, std::size_t length){
        std::istream in(&buf);
        std::cout << length << std::endl;
        std::cout << buf.size() << std::endl;
        in.ignore(length);
        boost::asio::async_read(socket_, buf, boost::asio::transfer_exactly(2), [this, self](boost::system::error_code ec, std::size_t length){
            std::istream in(&buf);
            std::cout << length << std::endl;
        });
    });
  }

  void do_write(std::size_t length)
  {
    auto self(shared_from_this());
    boost::asio::async_write(socket_, boost::asio::buffer(data_, length),
        [this, self](boost::system::error_code ec, std::size_t /*length*/)
        {
          if (!ec)
          {
            do_read();
          }
        });
  }

  tcp::socket socket_;
  enum { max_length = 1024 };
  char data_[max_length];
};

class server
{
public:
  server(boost::asio::io_service& io_service, short port)
    : acceptor_(io_service, tcp::endpoint(tcp::v4(), port)),
      socket_(io_service)
  {
    do_accept();
  }

private:
  void do_accept()
  {
    acceptor_.async_accept(socket_,
        [this](boost::system::error_code ec)
        {
          if (!ec)
          {
            std::make_shared<session>(std::move(socket_))->start();
          }

          do_accept();
        });
  }

  tcp::acceptor acceptor_;
  tcp::socket socket_;
};

int main(int argc, char* argv[])
{
  try
  {
    if (argc != 2)
    {
      std::cerr << "Usage: async_tcp_echo_server <port>\n";
      return 1;
    }

    boost::asio::io_service io_service;

    server s(io_service, std::atoi(argv[1]));

    io_service.run();
  }
  catch (std::exception& e)
  {
    std::cerr << "Exception: " << e.what() << "\n";
  }

  return 0;
}

Answer 1

The second async_read just reads 2 bytes. You explicitly say you want to "transfer exactly 2 bytes".

What you're looking at is that you do not /want/ to read 2 bytes, because you already did. You could adjust like:

size_t to_transfer = 2 - std::min(2ul, buf.size());
boost::asio::async_read(socket_, buf, boost::asio::transfer_exactly(to_transfer),

On a slightly related note, it can seem for the callback to not be invoked because you donot explicit flush std::cout. So adding

std::cout << std::unitbuf;

in main can remove that potential source of confusion.

DEMO

Live On Coliru

This sample is more careful in naming length (it's not length, it's bytes_transferred ) and dumping more values for buf.size() so you'll see what is really happening.

#include <boost/asio.hpp>
#include <cstdlib>
#include <iostream>
#include <memory>
#include <utility>

using boost::asio::ip::tcp;

class session : public std::enable_shared_from_this<session> {
  public:
    session(tcp::socket&& s) : socket_(std::move(s)) {}

    boost::asio::streambuf buf;

    void start() { do_read(); }

  private:
    void do_read() {
        auto self(shared_from_this());

        boost::asio::async_read_until(
            socket_, buf, "\r\n\r\n", [this, self](boost::system::error_code ec, std::size_t transferred) {
                std::cout << "async_read_until -> " << ec.message() << "\n";
                {
                    std::cout << "transferred: " << transferred << std::endl;
                    std::cout << "buffer: " << buf.size() << std::endl;
                    {
                        std::istream in(&buf);
                        in.ignore(transferred);
                    }
                    std::cout << "remaining buffer: " << buf.size() << std::endl;
                }

                size_t to_transfer = 2 - std::min(2ul, buf.size());
                boost::asio::async_read(socket_, buf, boost::asio::transfer_exactly(to_transfer),
                    [this, self](boost::system::error_code ec, std::size_t transferred) {
                        std::cout << "async_read -> " << ec.message() << "\n";
                        std::cout << "transferred: " << transferred << std::endl;
                        std::cout << "buffer: " << buf.size() << std::endl;
                        {
                            std::istream in(&buf);
                            char a, b;
                            if (in >> a >> b) {
                                std::cout << "Payload: '" << a << b << "'\n";
                            }
                        }
                        std::cout << "remaining buffer: " << buf.size() << std::endl;
                    });
            });
    }

    friend class server;
    tcp::socket socket_;
};

class server {
  public:
    server(boost::asio::io_service &io_service, short port) : acceptor_(io_service, tcp::endpoint({}, port)), socket_(io_service) {
        do_accept();
    }

  private:
    void do_accept() {
        acceptor_.async_accept(socket_, [this](boost::system::error_code ec) {
            std::cout << "async_accept -> " << ec.message() << "\n";
            if (!ec) {
                std::make_shared<session>(std::move(socket_))->start();
                //do_accept();
            }
        });
    }

    tcp::acceptor acceptor_;
    tcp::socket socket_;
};

int main() {
    std::cout << std::unitbuf;
    try {
        boost::asio::io_service io_service;

        server s(io_service, 6767);

        io_service.run();
    } catch (std::exception &e) {
        std::cerr << "Exception: " << e.what() << "\n";
    }
}

For example with a "client" like

echo -e 'hello\r\n\r\nmore to come' | netcat localhost 6767

The output looks like

async_accept -> Success
async_read_until -> Success
transferred: 9
buffer: 22
remaining buffer: 13
async_read -> Success
transferred: 0
buffer: 13
Payload: 'mo'
remaining buffer: 11