Receive partial data with timeout using TIdTcpClient

Question

How to receive a 100-byte-string with following conditions using TIdTcpClient ?:

• If nothing comes in, the Read call shall be blocking and thread will wait eternally
• If 100 bytes were received the Read call should return the byte string
• If more than 0 bytes, but less than 100 were received, Read call should return after some timeout (say 1 second) in order to return at least something in a reasonable time, without producing timeout exception, because exception handling in Delphi IDE's debug mode hasn't been made convenient.

My not optimal code for now is as follows:

unit Unit2;

interface

uses
  System.Classes, IdTCPClient;

type
  TTcpReceiver = class(TThread)
  private
    _tcpc: TIdTCPClient;
    _onReceive: TGetStrProc;
    _buffer: AnsiString;
    procedure _receiveLoop();
    procedure _postBuffer;
  protected
    procedure Execute(); override;
  public
    constructor Create(); reintroduce;
    destructor Destroy(); override;
    property OnReceive: TGetStrProc read _onReceive write _onReceive;
  end;

implementation

uses
  System.SysUtils, Vcl.Dialogs, IdGlobal, IdExceptionCore;

constructor TTcpReceiver.Create();
begin
  inherited Create(True);
  _buffer := '';
  _tcpc := TIdTCPClient.Create(nil);
  //_tcpc.Host := '192.168.52.175';
  _tcpc.Host := '127.0.0.1';
  _tcpc.Port := 1;
  _tcpc.ReadTimeout := 1000;
  _tcpc.Connect();
  Suspended := False;
end;

destructor TTcpReceiver.Destroy();
begin
  _tcpc.Disconnect();
  FreeAndNil(_tcpc);
  inherited;
end;

procedure TTcpReceiver.Execute;
begin
  _receiveLoop();
end;

procedure TTcpReceiver._postBuffer();
var buf: string;
begin
  if _buffer = '' then Exit;
  buf := _buffer;
  _buffer := '';
  if Assigned(_onReceive) then begin
    Synchronize(
      procedure()
      begin
        _onReceive(buf);
      end
    );
  end;
end;

procedure TTcpReceiver._receiveLoop();
var
  c: AnsiChar;
begin
  while not Terminated do begin
    try
      c := AnsiChar(_tcpc.IOHandler.ReadByte());
      _buffer := _buffer + c;
      if Length(_buffer) > 100 then
        _postBuffer();
    except
      //Here I have to ignore EIdReadTimeout in Delphi IDE everywhere, but I want just to ignore them here
      on ex: EIdReadTimeout do _postBuffer();
    end;
  end;
end;

end.

Answer 1

TCP is stream oriented, not message oriented like UDP is. Reading arbitrary bytes without any structure to them is bad design, and will easily corrupt your communications if you stop reading prematurely and then the bytes you wanted to read arrive after you have stopped reading. The bytes are not removed from the socket until they are read, so the next read may have more/different bytes than expected.

If you are expecting 100 bytes, then just read 100 bytes and be done with it. If the sender only sends 50 bytes, it needs to tell you that ahead of time so you can stop reading after 50 bytes are received. If the sender is not doing that, then this is a very poorly designed protocol. Using a timeout to detect end-of-transmission in general is bad design. Network lag could easily cause false detections.

TCP messages should be adequately framed so that the receiver knows exactly where one message ends and the next message begins. There are three ways to do that in TCP:

1. use fixed-length messages. The receiver can keep reading until the expected number of bytes has arrived.

2. send a message's length before sending the message itself. The receiver can read the length first and then keep reading until the specified number of bytes has arrived.

3. terminate a message with a unique delimiter that does not appear in the message data. The receiver can keep reading bytes until that delimiter has arrived.

---

That being said, what you are asking for can be done in TCP (but shouldn't be done in TCP!). And it can be done without using a manual buffer at all, use Indy's built-in buffer instead. For example:

unit Unit2;

interface

uses
  System.Classes, IdTCPClient;

type
  TTcpReceiver = class(TThread)
  private
    _tcpc: TIdTCPClient;
    _onReceive: TGetStrProc;
    procedure _receiveLoop;
    procedure _postBuffer;
  protected
    procedure Execute; override;
  public
    constructor Create; reintroduce;
    destructor Destroy; override;
    property OnReceive: TGetStrProc read _onReceive write _onReceive;
  end;

implementation

uses
  System.SysUtils, Vcl.Dialogs, IdGlobal;

constructor TTcpReceiver.Create;
begin
  inherited Create(False);
  _tcpc := TIdTCPClient.Create(nil);
  //_tcpc.Host := '192.168.52.175';
  _tcpc.Host := '127.0.0.1';
  _tcpc.Port := 1;
end;

destructor TTcpReceiver.Destroy;
begin
  _tcpc.Free;
  inherited;
end;

procedure TTcpReceiver.Execute;
begin
  _tcpc.Connect;
  try
    _receiveLoop;
  finally
    _tcpc.Disconnect;
  end;
end;

procedure TTcpReceiver._postBuffer;
var
  buf: string;
begin
  with _tcpc.IOHandler do
    buf := ReadString(IndyMin(InputBuffer.Size, 100));
  { alternatively:
  with _tcpc.IOHandler.InputBuffer do
    buf := ExtractToString(IndyMin(Size, 100));
  }
  if buf = '' then Exit;
  if Assigned(_onReceive) then
  begin
    Synchronize(
      procedure
      begin
        if Assigned(_onReceive) then
          _onReceive(buf);
      end
    );
  end;
end;

procedure TTcpReceiver._receiveLoop;
var
  LBytesRecvd: Boolean;
begin
  while not Terminated do
  begin
    while _tcpc.IOHandler.InputBufferIsEmpty do
    begin
      _tcpc.IOHandler.CheckForDataOnSource(IdTimeoutInfinite);
      _tcpc.IOHandler.CheckForDisconnect;
    end;

    while _tcpc.IOHandler.InputBuffer.Size < 100 do
    begin
      // 1 sec is a very short timeout to use for TCP.
      // Consider using a larger timeout...
      LBytesRecvd := _tcpc.IOHandler.CheckForDataOnSource(1000);
      _tcpc.IOHandler.CheckForDisconnect;
      if not LBytesRecvd then Break;
    end;

    _postBuffer;
  end;
end;

end.

---

On a side note, your statement that " exception handling in Delphi IDE's debug mode hasn't been made convenient " is simply ridiculous. Indy's IOHandler has properties and method parameters for controlling exception behavior, and also if you don't like the way the debugger handles exceptions then simply configure it to ignore them. You can configure the debugger to ignore specific exception types, or you can use breakpoints to tell the debugger to skip handling exceptions in specific blocks of code.