Boost::asio async_read simple file upload

Question

I'm trying to create a simple file upload using boost::asio and ajax. Somehow the bytes_transferred reaches a maximum at 21845 (or something like that) even though my buffer size is 500MB. With text files std::cout displays something but with binary files it cuts off right after the header (still 21845 bytes are transferred).

I tried using different methods like async_read_some and async_read_until but it didn't change anything. Also tried socket.set_option() with keep-alive and changing socket buffer size.

class tcp_connection
{
public:
void start() {
    boost::asio::async_read(socket, boost::asio::buffer(buf), boost::asio::transfer_at_least(1), boost::bind(&tcp_connection::handler, error, bytes_transferred));
}
private:
boost::array<unsigned char, 500000000> buf;
void handler(error, bytes) {
    std::cout << buf.data() << std::endl;

    boost::asio::async_write(socket, response_buffer, boost::bind(&tcp_connection, handler));
}
}

Please look at https://pastebin.com/dPvVGsjU for the full minimal-reproductible example

I don't receive error messages, although I am ignoring eof errors.

Answer 1

Why do you suppose that the whole data will be read during one read operation ? Your completion condition is transfer_at_least(1) , so read operation can read 1, 10, 100 or N bytes and return. You should read data until the whole content is received. Because you don't know the length of input data, you continue reading until eof occures.

So logic of handler passed into async_read should be:

void send_response2(const boost::system::error_code& err, const size_t& bytes) 
{
    if (err == boost::asio::error::eof)
    {
        // analyze your buf2 here if it is what you wanted to read
        // here you can start sending data
    }
    else if (err)
    { /* problem */ }
    else 
    {
        // call again async_read with send_response like
        boost::asio::async_read(socket, boost::asio::dynamic_buffer(buf2), boost::asio::transfer_at_least(1), boost::bind(&tcp_connection::send_response2, shared_from_this(),
                                                boost::asio::placeholders::error,
                                                boost::asio::placeholders::bytes_transferred));
    }
}   

---

Instead of buffer::array as buffer with fixed size, you can use dynamic buffer . Then your buffer will be extended, by adding new read bytes.

std::string buf2; // as data member
// then pass buffer by
boost::asio::dynamic_buffer(buf2)