How do I receive socket data from multiple reads?

Question

I am tasked with working on a legacy D6 application that uses TClientSocket to request data from a TCP/IP Server which returns XML for which, I am required to integrate with a new 3rd party server and I am having an intermittent problem receiving the full data sent in the Server's response. When this happens I get the XML string over multiple OnRead events despite that the XML is pretty small, about 1.76 KB.

The structure of the response is that the first four bytes return indicate:

Byte Position, type/purpose:

0 - 0x02 (STX)
1 - Length, LSB
2 - Length
3 - Length, MSB
Bytes 4+ are the xml payload

However, the programmer before me simply used Socket.ReceiveText because all of the response being received for existing logic was very small, well under 200 bytes in most cases...essentially success confirmation or error data.

Would someone mind terribly giving me an idea of how I can successfully consume the response taking into account that I'm getting it in chunks? I've not used TClientSocket/TServerSocket much even when I was using Delphi regularly and I cannot change to something I am more familiar with.

Thanks in advance.

Answer 1

In the OnRead event, read whatever bytes are currently available on the socket and append them to a buffer. Then you can loop through that buffer extracting only complete XML messages and process them as needed, leaving incomplete XML messages in the buffer so they can be completed in later OnRead events.

For example:

type
  XmlHdr = packed record
    Stx: Byte;
    XmlLen: array[0..2] of Byte;
  end;

procedure TForm1.ClientSocket1Connect(Sender: TObject; Socket: TCustomWinSocket);
begin
  // allocate the receive buffer
  Socket.Data := TMemoryStream.Create;
end;

procedure TForm1.ClientSocket1Disconnect(Sender: TObject; Socket: TCustomWinSocket);
begin
  // free the receive buffer
  TMemoryStream(Socket.Data).Free;
  Socket.Data := nil;
end;

procedure TForm1.ClientSocket1Read(Sender: TObject; Socket: TCustomWinSocket);
var
  Strm: TMemoryStream;
  RecvLen: Integer;
  StrmSize: Int64;
  Ptr: PByte;
  hdr: XmlHdr;
  xml: AnsiString;
begin
  Strm := TMemoryStream(Socket.Data);

  // check how many bytes are currently available on the socket
  RecvLen := Socket.ReceiveLength;
  if RecvLen <= 0 then Exit;

  // read the bytes, appending them to the end of the buffer

  StrmSize := Strm.Size;
  Strm.Size := StrmSize + RecvLen;

  Ptr := PByte(Strm.Memory);
  Inc(Ptr, StrmSize);

  RecvLen := Socket.ReceiveBuf(Ptr^, RecvLen);
  if RecvLen <= 0 then
  begin
    Strm.Size := StrmSize;
    Exit;
  end;

  Strm.Size := StrmSize + RecvLen;

  // loop through the buffer processing only complete XML messages

  Strm.Position := 0;    
  while (Strm.Size - Strm.Position) >= SizeOf(hdr) do
  begin
    // make sure the next byte starts a new header
    Strm.ReadBuffer(hdr.Stx, 1);
    if Hdr.Stx <> $2 then Continue;    

    // read the header's XML length
    Strm.ReadBuffer(hdr.XmlLen[0], 3);

    {
    0 - Length, LSB
    1 - Length
    2 - Length, MSB
    }
    RecvLen := (Integer(hdr.XmlLen[2]) shl 16) or (Integer(hdr.XmlLen[1]) shl 8) or Integer(hdr.XmlLen[0]);

    // check if the complete XML has been received
    if (Strm.Size - Strm.Position) < RecvLen then
    begin
      // nope, keep waiting
      Strm.Seek(-SizeOf(hdr), soCurrent);
      Break;
    end;

    // extract the complete XML
    SetLength(xml, RecvLen);
    Strm.ReadBuffer(PAnsiChar(xml)^, RecvLen);    

    // process xml as needed...
  end;

  if strm.Position > 0 then
  begin
    // remove consumed bytes from the buffer and compact it
    StrmSize := Strm.Size - Strm.Position;
    if StrmSize = 0 then
    begin
      Strm.Clear;
    end else
    begin
      Ptr := PByte(Strm.Memory);
      Inc(Ptr, Strm.Position);
      Move(Ptr^, Strm.Memory^, StrmSize);
      Strm.Size := StrmSize;
    end;
  end;
end;