EOF with boost::asio::read

Question

I am having a small problem with my client/server application in C++. It uses boost::asio for remote communication and protocol buffers for serialization. Here the client:

// Time to write
char sizeMessage[20];
string content = request.SerializeAsString();
size_t request_length = content.size();            

boost::asio::write(s, boost::asio::buffer(content.c_str(), request_length));
boost::system::error_code error;

boost::asio::read(s, boost::asio::buffer(sizeMessage, 20));

int bufferSize = atoi(sizeMessage);
char* responseString = new char[bufferSize];        
size_t reply_length = boost::asio::read(s, boost::asio::buffer(responseString, bufferSize), boost::asio::transfer_all(), error);
if(!error){
    response.ParseFromArray((const void*)responseString, reply_length);
    success = true;                
}else{
    response.set_status(ExecResponse::ERROR);                
}


delete[] responseString;

In this particular scenario of my application, the server's response can vary in size, going from a few bytes up to some MBs. For that reason the server first writes the size of the output which I then use to allocate the right amount of memory for my buffer. For some reason I get an error when reading the data (the size is always correctly read). A little bit of debugging revealed asio.misc:2 which by googling I figured out it is EOF (effectively the number of read bytes is smaller than the buffer size). Here the server side:

ExecResponse response;
ExecutionServerHandler execHandler(this->sharedData);
execHandler.process(rd, request, response);   
string output = response.SerializeAsString();
//First write the length
ostringstream stringLength;
stringLength << output.size();
response.PrintDebugString();
string lengthString = stringLength.str();    
size_t bw = ba::write(bsocket_, ba::buffer(lengthString));
bw = ba::write(bsocket_, ba::buffer(output));

This is actually my first time working with boost so I am not an experienced developer. Moreover, it is important to mention that in other parts of my application, this mechanism works but my other messages have some differences: although they are still variable in size, they are always small (a few hundred bytes). The server side is identical but in the client side I am using boost::asio::transfer_at_least(1) and I do not write the size in advance. If I apply those changes to this case I noticed the read call is able to retrieve up to 65536 bytes and no more (returning EOF prematurely). My development machines are Debian 5 and Ubuntu 10.04 for 64 bits and I get the same behaviour for Boost 1.40 and 1.47. If somebody realizes I am doing something wrong, I would appreciate to point it out. I am desperated trying to make it work.

Answer 1

Looks to me like there could be a problem in your protocol: How does your client know the length of lengthString? You are converting an arbitrary number into a string and then writing that to the buffer, without padding or anything? Without seeing more of the code it is difficult to see, but this definitely looks suspicious.

Typically when having variable length messages, you define some kind of packet format eg the length field is 4 bytes ( choose the correct number of bytes for your messages ) followed by the message and then you can send the (in this case) int to the client which will firstly read 4 bytes, and then know how many more bytes to read from the socket. You might want to look at functions like htons , etc.

If you have a variable length header, you would have some kind of delimiter to read up to. This type of header is illustrated in the asio HTTP examples where "\\r\\n\\r\\n" is the delimiter used. This example might also be useful to you since as reima stated, you would then need to change your client to first read the header and then the message body.

Answer 2

It seems that you are sending a request to the server to ask the size of the upcoming reply, but because TCP streams have no delimiter in themselves you might be receiving the size together with a part or the whole of the reply while expecting the size only! You should take that part of the data into consideration before attempting to read the remainder of the reply (and obviously wait for less bytes).