How to manage large number of TCP connections using ASP.NET and C#

Question

I have an application which connects to a third party server let's call it Server-A. I have been given four different ports ie 4000, 40001, 40002, 40003. On each port I can create 20 connections so I can create 80 total connections with server-A. I want to create a service layer that should communicate with server-A on mentioned ports. The technology will be asp.net C#.

The problem statement

1- Application should be non-blocking/asynchronous to entertain 10 to 20 million request per day

2- Whenever the service layer starts it create 20 connections on each port. (Total 80 connections)

2- All connections should remain connected/alive 24/7 and reconnect whenever any connections drops/disconnects. It will send a heartbeat message in idle time.

My Questions

• How can I manage these connection? Should I add those to a static list one by one when a TCP socket is successful?
• How can I know that a certain connection is dropped/disconnected?
• How can I send certain requests on different ports? Let's say if a>b send it on port 4000 else if a<=b send it on 4001
• How can I make it asynchronous?

For an initial start I created a single TCP connection on single port and it works as expected. Then I replicated the same code for other port, but I know it is very bad approach and I have to copy same code 80 times to make 80 connections. I want a clean and scalable way to achieve it, so that in future may be I increase the connection to 100 or more. Is there any framework which I can use? Any help would be greatly appraised.

Answer 1

To handle such a large volume of traffic you need to do a few things.

Assumptions

• You are connecting to another client's server.
• You have a large volume of web traffic from either multiple machines or from multiple working processes on any given machine.
• You know how to create TCP client server objects and handle the connections.

For less than 80 worker threads across your servers:

• Because each thread processes synchronously, you only need to use a single connection for each thread.
• If no single web server is running more than 20 worker processes, then you can designate a single port for each server to use. Stick the port in your web.config file as a variable and use that when creating connections. You will never hit the limit.
• Store your connection in a shared object that the entire app can use (could put this in your BLL layer) and if you have a connection error, re-create a new connection on that thread.

For more than 80 worker threads across your servers:

• Do the same as the last step but at this point you either need to negotiate for more connections or you will add a new layer in between your application and the server you wish to reach.
• This second layer acts as a broker for the two sides and can manage a pool of connections instead and draws off a connection each time you need to access Server-A and puts it back into the pool when finished.
• Anytime you connect to the broker application, spawn a new thread to do the processing until the connection is dropped or closed.
• Keep track of your open connections and viola you can have as many clients as you need but your bottleneck will be those 80 connections out even if you have hundreds or thousands in.

Answer 2

@Kartoos Khan, i have made some services with those requirements and using asynchronous methods is the best way to create high performance services in C#, because:

• It does not block IO peripherals, as can be sockets.
• Minimize the threads and improve the performance to it.

Let me recommend you the book Writing High-Performance .NET Code . The chapter 4, Asynchronous Programming has the information that you need to know to improve the performance.

In my experience those are my recommendations:

1. Create a main threat to handle the main program.

2. Create a class to handle the Socket Server, which implements an asynchronous process to accept connections, using the methods BeginAccept and EndAccept , here is a sample of how to use it. Create another class to handled the socket connections, which has as a property the Socket object.

   2.1 create a method to start the Reading process, which will be called by the Server class to start the communication between the endpoints. This methos will start the process of read in an asynchronous way.

   2.2 To read and write in an asyncrhonous way, it is necessary to get the NetworkStream from the socket, and use the methods BeginRead and EndRead , to receive data, and BegineWrite and EndWrite to send data. Here there is the documentation.

In the case that your service only needs to connect to a Server, ignore the step 1and implement the Client class to start the connection to an specific EndPoint.

3. Use a collection class, as can be a Dictionary , Key-Value-Pair collection, to store each Client Class and use the socket ID as the key to access to each Client Class.

4. Due each Client Socket handles it own socket, i use to implements a way to reconnect at the same Client Socket, i this way each Client is responsable for itself.

5. The main program will be responsable to create each Client Server and set the EndPoint of each client, as you need, and start to connect each of them. In this case, TCPClient allow you begin an asynchronous process for connect, using the methods BeginConnect and EndConnect .

Here you can see more details about this issue.

I hope this might be useful for you.