FTP error: 530 Sorry, Only 5 connections per host are allowed

Question

I have a FTP solution that works great. However after a while, perheps 20-30 minutes, I can get this error:
530 Sorry, Only 5 connections per host are allowed

I have googled on this problem but not really finding any solution to it. I have different threads calling the function but only allow the function to be called maximum each 2 seconds. I am make use of: using statements but I think I don't want to use:

 request.KeepAlive = false;

to be sure that the connection will be closed. As I before was told to have it open in "ServicePoint" as I use "ConnectionGroupName" because open up a new connection is resource expensive and it is smoother to keep it open.

I think but are not sure. Please tell me if that is a bad approach. But when this error occurs. I like to close all connections, in this case 5 connections so it starts over from Zero connections. Is this possible to do and how would I go about to do that in code since I am not sure how to call those connections in the code?

 public async Task<List<String>> FtpGetFiles(String host, String folderpath, String user_name, String password) { List<String> serverfileLIST = new List<String>(); String error = ""; try { var request = (FtpWebRequest)WebRequest.Create(host + "/" + folderpath + "/"); request.Credentials = new NetworkCredential(user_name, password); request.Method = WebRequestMethods.Ftp.ListDirectory; request.ConnectionGroupName = host.Replace(".", "").Replace(":", "").Replace("/", "").Replace("-", "").Replace("_", "") + user_name; request.ServicePoint.ConnectionLimit = 4; request.Timeout = 20000; using (var responseWeb = await request.GetResponseAsync()) { var response = (FtpWebResponse)responseWeb; if (response.StatusDescription.Contains("150") || //150 opening binary mode data connection for file list response.StatusDescription.Contains("226")) { using (StreamReader streamReader = new StreamReader(response.GetResponseStream())) { while (streamReader.Peek() >= 0) { serverfileLIST.Add(streamReader.ReadLine()); } } return serverfileLIST; } else { error = response.StatusDescription; } } } catch (WebException ex) { String status = ((FtpWebResponse)ex.Response).StatusDescription; error = status; } return serverfileLIST; }

EDIT:
I have attached a network.txtfile. I can see in my own logfile that at this time GMT, I got the below error:
09 Dec 2019 16:09:40 530 Sorry, Only 5 connections per host are allowed
09 Dec 2019 16:09:42 530 Sorry, Only 5 connections per host are allowed

The network.txtfile is attached which is GMT also in the link below and I wonder if it is possible to trace what the problem is and how it could be solved?
network.txt

I could find those lines in the txt file where it occurs:
System.Net Information: 0 : [14868] FtpControlStream#13919402 - Sending command [PASS ********] System.Net Information: 0 : [14868] FtpControlStream#13919402 - Received response [530 Sorry, Only 5 connections per host are allowed] System.Net Information: 0 : [14868] FtpWebRequest#45677341::(Releasing FTP connection#13919402.) System.Net Error: 0 : [14868] Exception in FtpWebRequest#45677341::EndGetResponse - The remote server returned an error: (530) Not logged in.. at System.Net.FtpWebRequest.EndGetResponse(IAsyncResult asyncResult)

Answer 1

Just the maximum throughput of one call per every 2 seconds does not prevent the external threads from invoking more than 5 connections. Just imagine you will temporarily have a very slow connection to the server or the list you request will be very long. It could take more time to download than usual (let's say it will take 15 seconds for some reason). Because it will take more time to finish, you will eventually reach the cap of 5 concurrent connections and get the error, because you will be initiating new calls before the previous downloads finished.

You could use SemaphoreSlim to "queue" all the calls to your method FtpGetFiles . This would make sure your threads are not executing more than 5 concurrent calls. When one call finishes, semaphore would release a spot and another call would start.

It would look something like this:

public class TaskQueue
{
    private readonly SemaphoreSlim semaphore;
    private int pendingTasks;
    private int runningTasks;

    public TaskQueue(int maxConcurrencyLevel = 5)
    {
        semaphore = new SemaphoreSlim(maxConcurrencyLevel);
    }

    private async Task AddTask(Func<Task> task)
    {
        Interlocked.Increment(ref pendingTasks);
        await semaphore.WaitAsync();

        Interlocked.Decrement(ref pendingTasks);
        Interlocked.Increment(ref runningTasks);

        try
        {
            await task();
        }
        finally
        {
            Interlocked.Decrement(ref runningTasks);
            semaphore.Release();
        }
    }
}

Every thread calling your method would call this:

TaskQueue.AddTask(async () => await FtpGetFiles(host, folderpath, username, password));