![](/img/trans.png)
[英]Download multiple files concurrently from FTP using FluentFTP with a maximum value
[英]Azure function to copy files from FTP to blob storage using FluentFTP
我有一個添加每日文件的 FTP 源,我需要使用 azure 函數中的 FluentFTP 庫每天將文件從 FTP 復制到 blob 存儲
我在 Azure 函數中使用 C#,我完成了所有編碼部分,但沒有從 FTP 下載文件以將其直接復制到 blob 目標。
//#r "FluentFTP"
#r "Newtonsoft.Json"
#r "System.Data"
#r "Microsoft.WindowsAzure.Storage"
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Collections.Generic;
using System.Globalization;
using System;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;
using FluentFTP;
public static async Task<IActionResult> Run(HttpRequest req, ILogger log)
{
string blobConnectionString = "ConnectionString";
// Gestione BLOB Storage
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(blobConnectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("out");
using (FtpClient conn = new FtpClient())
{
conn.Host = "ftp server";
conn.Credentials = new NetworkCredential("username", "pass");
// get a list of files and directories in the "/OUT" folder
foreach (FtpListItem item in conn.GetListing("/OUT"))
{
// if this is a file and ends with CSV
if (
item.Type == FtpFileSystemObjectType.File
&&
item.FullName.ToLower().EndsWith(".csv")
)
{
string yyyy = item.FullName.Substring(10,4);
string mm = item.FullName.Substring(14,2);
string dd = item.FullName.Substring(16,2);
var fileName = "out/brt/" + yyyy + "/"+ mm + "/"+ dd + "/" + item.Name;
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
// download the file
conn.DownloadFile( blockBlob , item.FullName);
}
}
return new OkObjectResult($"Hello");
}
}
如果我可以使用 blob 容器作為 FluentFTP 函數的目標,那么這將是最好的,但是這樣我得到的錯誤是我正在使用的這個 blob 塊不是目標
這是我得到的錯誤
無法從“Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob”轉換為“string”
我不知道是否有另一種方法可以在本地下載文件,以便我可以將其上傳到 blob 而不是使用這個client.DownloadFile
函數。
如果要使用 FluentFTP,可以使用以下兩種方法之一獲取 Blob 上傳流:
然后你可以使用FTPClient.Download
方法,它接受一個Stream
public bool Download(Stream outStream, string remotePath, IProgress<double> progress = null)
像這樣的東西:
[FunctionName("HttpTriggerCSharp")]
public static async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
var ftpHost = "xxxxx";
var ftpUserName = "xxxx";
var ftpPassword = "xxxx";
var filename = "xxxxx";
string blobConnectionString = "xxxxxxxxxxx";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(blobConnectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("xxxxx");
FtpClient client = new FtpClient(ftpHost, ftpUserName, ftpPassword); // or set Host & Credentials
client.EncryptionMode = FtpEncryptionMode.Implicit;
client.SslProtocols = SslProtocols.None;
client.ValidateCertificate += new FtpSslValidation(OnValidateCertificate);
client.Connect();
void OnValidateCertificate(FtpClient control, FtpSslValidationEventArgs e) {
// add logic to test if certificate is valid here
e.Accept = true;
}
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
var outStream = blockBlob.OpenWrite();
client.Download(outStream,filename);
outStream.Commit(); // I'm not sure if this is needed?
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
return (ActionResult)new OkObjectResult($"Action succeeded.");
}
Fluent FTP 可能無法實現。
有采用Stream
的FtpClient.Download
方法。
public bool Download(Stream outStream, string remotePath, IProgress<double> progress = null)
但看起來沒有 API 來獲取"blob upload Stream" 。
相反,您無法從 FluentFTP 獲得“FTP 下載流” (您可以將其與 blob API 一起使用)。
但是你可以使用原生的 .NET FtpWebRequest
FTP 客戶端,它有 API 來獲取“FTP 下載流” :
public static async System.Threading.Tasks.Task RunAsync([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
{
var ftpUserName = "xxxxxx";
var ftpPassword = "xxxxxxxxx";
var filename = "test.png";
string blobConnectionString = "xxxxxxx";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(blobConnectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("myblobcontainer");
FtpWebRequest fileRequest = (FtpWebRequest)WebRequest.Create("ftp://xxxxxx/" + filename);
fileRequest.Method = WebRequestMethods.Ftp.DownloadFile;
fileRequest.Credentials = new NetworkCredential(ftpUserName, ftpPassword);
FtpWebResponse fileResponse = (FtpWebResponse)fileRequest.GetResponse();
Stream fileStream = fileResponse.GetResponseStream();
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
await blockBlob.UploadFromStreamAsync(fileStream);
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
}
您不能使用 SDK 直接將文件從 FTP 移動到 Blob。 您必須先臨時下載文件 - 下載到流中(取決於它們的大小)或臨時文件。
但是,如果您只想按計划將文件從 FTP 移動到 Blob,我實際上會考慮使用專為此類任務構建的 Azure 數據工廠: https : //docs.microsoft.com/en-我們/天藍色/數據工廠/連接器-ftp
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.