如何从功能管理与sftp的并发连接

Question

How do we manage concurrent connections, perhaps by batching the files into the same connection?

I'm creating an azure function to be triggered by new blobs (this might be changed to a simple timer-trigger). An example of a blob would be:

{
"server":"sftp.mysftpserver.com",
"credentials":"topsecret",
"body":"This text will be uploaded to the sftp server above"
}

The function will login to the server, and send the payload in the body .

How do we limit the number of connections that are being initiated to the same server?

We might have 500 blobs that are all triggering the function, which in turn is opening 500 connections to: sftp.mysftpserver.com .

At the same time there might be just 2 blobs that are initiating connections to sftp.otherserver.net .

Answer 1

Without more info on the specific implementation I can only give general advice. Assuming you are using some sort of FTP client, you can use a static instance of that client to share it within a Functions instance. If throughput is that high then the Functions service will scale to multiple instances, but assuming you are on the consumption plan this will max out at 200 . Unless there is a huge bottleneck, the a 500 document spike would likely be cleared before the scaling got anywhere near 200 instances.

For example you would do something like this:

// Create a single, static SftpClient
private static SftpClient client = new sftpClient(endpoint, user, pass);

public static async Task Run(string input)
{
    var response = await sftpClient.SendAsync("filename.txt", input);
}

If you aren't using an SFTP library that can do this by default, you can create a static helper class that handles creating the client.