C# Doing retry on tcp socket communication

Question

I'm trying to do a retry coding if my client fail to connect to my server. Below is how i do:

In main function:

Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

client.BeginConnect(remoteEP, new AsyncCallback(ConnectCallback), client);

ConnectCallback:

private void ConnectCallback(IAsyncResult ar)
{
    try
    {
        // Retrieve the socket from the state object.  
        Socket client = (Socket)ar.AsyncState;

        // Complete the connection.  
        client.EndConnect(ar);
        _logger.Info("## Connection to server successful at " + strServerIP + ":" + strServerPort);
    }
    catch (Exception ex)
    {
        _logger.Info("## Connection to server failed. Retrying...");

        Socket client = (Socket)ar.AsyncState;
        IPEndPoint remoteEP = new IPEndPoint(IPAddress.Parse(strServerIP), Convert.ToInt32(strServerPort));

        client.BeginConnect(remoteEP, new AsyncCallback(ConnectCallback), client);
    }
}

I will catch the exception in ConnectCallback when connection fail and do retry.

But I found out that, if it retried 10 times, when server is up, the server will get 10 connection from that same client. If it retry 50 times, when server is up, then server will get 50 connections.

Is my coding something wrong? Seems like everytime it retry, my server will get a new connection.

Answer 1

Without a functioning example it's difficult to know. If this is close to what you are actually doing, I suspect a couple of things are wrong. The Socket object appears to be blocking by default, but something is generating your exception and it may not be what you think it is. First thing to do is only catch the SocketException and then only retry when the exception represents something that might indicate a retry would work. Put in a delay because if it didn't work 1 ms ago, it probably won't work now. Put in a sanity counter so it gives up retrying after so many tries. Examine your protocol to make sure you are sending the server what it expects. Above all Close your sockets.

What I suspect you are seeing is a bunch of Socket connections caused by an exception (which may or may not be related to the sockets). Since you never close them they just build up. I suspect that eventually the GC might kick in and run finalizers on the objects which would then drop their connections. More likely the server would drop the connections. Either way if you aren't explicitly closing your Socket then it will hang around until something times out.

Here is a working example that demonstrates what I think you are asking. Again you need to decide under what conditions you should retry because retrying if anything goes wrong is not good. It may cause your program to constantly churn threads and possibly even connections.

using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
using Microsoft.Extensions.Logging;

namespace socketTst {
class Program {
    static ILoggerFactory loggerFactory = new LoggerFactory().AddConsole().AddDebug();
    static ILogger _logger;
    static AutoResetEvent finish = new AutoResetEvent(false);
    static String Hostname = "www.google.com";
    static int Port = 80;
    static int RetryCount = 0;

    static void ConnectCallback(IAsyncResult ar) {
        _logger.LogInformation($"## ConnectCallback entered");
        // Retrieve the socket from the state object.  
        Socket client = (Socket) ar.AsyncState;
        try {
            // Complete the connection.  
            client.EndConnect(ar);
            var s = new byte[] { 1 };
            client.Send(s);

            var buf = new byte[1024];
            var cnt = client.Receive(buf);

            _logger.LogInformation($"## Connection to server successful at {client.RemoteEndPoint}");
            if (cnt > 0) {
                var returned = Encoding.UTF8.GetString(buf, 0, cnt);
                _logger.LogInformation($"## Data returned: {returned}");
                }
            else {
                _logger.LogInformation($"## No data returned");
                }
            finish.Set(); // signal end of program
            }
        catch (SocketException sockExcep) {
            _logger.LogInformation($"## Exception: {sockExcep.Message}");
            _logger.LogInformation("## Connection to server failed. Retrying...");
            // This is a bad idea.  You don't know what is wrong so retrying might not be useful.
            // What if this is an unknown host or some other error that isn't likely to be
            // resolved by a retry ???
            RetryCount++;
            if (RetryCount > 10) {
                _logger.LogInformation("## Not able to reach host after 10 tries");
                finish.Set(); // signal end of program
                return; // give up
                }
            Thread.Sleep(797); // wait a bit
            var dest = new DnsEndPoint(Hostname, Port);
            client.BeginConnect(dest, new AsyncCallback(ConnectCallback), client);
            }
        catch (Exception ex) {
            _logger.LogInformation($"## Exception: {ex.Message}");
            }
        _logger.LogInformation($"## ConnectCallback exited");
        }

    static void Main(string[] args) {
        _logger = loggerFactory.CreateLogger<Program>();
        Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
        client.Blocking = true;
        var dest = new DnsEndPoint(Hostname, Port);

        _logger.LogInformation($"Attempting connection to {dest.Host}:{dest.Port}");
        _logger.LogInformation($"Socket blocking: {client.Blocking}");

        _logger.LogInformation("Calling BeginConnect");
        var thd = client.BeginConnect(dest, new AsyncCallback(ConnectCallback), client);
        _logger.LogInformation("BeginConnect complete");

        _logger.LogInformation("Calling WaitOne");
        finish.WaitOne(); // don't let program end until connection is made
        _logger.LogInformation("WaitOne complete");

        client.Close();
        Thread.Sleep(25); // if you don't do this the program ends before all the log output can be written
        Console.WriteLine("Program complete");
        }
    }
}

I've tested this code using .NET Core 2.1 and you'll need the following nuget packages to run it:

Microsoft.Extensions.Logging
Microsoft.Extensions.Logging.Console
Microsoft.Extensions.Logging.Debug"

Successful execution looks like this:

info: socketTst.Program[0]
      Attempting connection to www.google.com:80
info: socketTst.Program[0]
      Socket blocking: True
info: socketTst.Program[0]
      Calling BeginConnect
info: socketTst.Program[0]
      BeginConnect complete
info: socketTst.Program[0]
      Calling WaitOne
info: socketTst.Program[0]
      ## ConnectCallback entered
info: socketTst.Program[0]
      ## Connection to server successful at 172.217.15.68:80
info: socketTst.Program[0]
      ## Data returned: HTTP/1.0 400 Bad Request
      Content-Length: 54
      Content-Type: text/html; charset=UTF-8
      Date: Wed, 26 Sep 2018 03:32:39 GMT

      <html><title>Error 400 (Bad Request)!!1</title></html>
Program complete