I tried to download files from site which is support TLS 1.3. I got error below.
System.Net.Http.HttpRequestException: 'The SSL connection could not be established, see inner exception.'
AuthenticationException: Authentication failed because the remote party sent a TLS alert: 'DecryptError'.
Also I tried to add: ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls13 | SecurityProtocolType.Tls12;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls13 | SecurityProtocolType.Tls12;
Also I tried other protocols but site is supporting tls 1.3 as i said above. By the way, when I remove parallel foreach and run its working well. However, I would like to download as a bulk.
I used Parallel.ForEach could it be related to this?
foreach (var directory in directories)
{
if (directory.Contains($"C:\\Websites\\{domain}\\Uploads") || directory.Contains($"C:\\Websites\\{domain}\\CustomCss"))
{
if (directory != null)
{
var directoryName = directory.Replace($"C:\\Websites\\{domain}\\", "/"); //String.Empty
if (directoryName.Contains("\\"))
{
directoryName = directoryName.Replace("\\", "/");
}
var createDirectory = await directoryClient.CreateDirectory(new CreateDirectoryRequest { FolderName = directoryName });
Parallel.ForEach(Directory.GetFiles(directory), new ParallelOptions { MaxDegreeOfParallelism = 15 }, async (item) =>
{
if (item != null)
{
var ms = new MemoryStream();
using (FileStream file = new FileStream(item, FileMode.Open, FileAccess.ReadWrite))
{
using (ms = new MemoryStream())
{
file.CopyTo(ms);
ms.ToArray();
}
}
string filename = "";
Uri uri = new Uri(item);
if (uri.IsFile)
{
filename = System.IO.Path.GetFileName(uri.LocalPath);
}
UploadFileResponse uploadResponse = new UploadFileResponse();
if (isImage(MimeMapping.MimeUtility.GetMimeMapping(item)))
{
byte[] imageContent = await DownloadAndCompareImages(domain, directoryName + "/" + filename, "90");
uploadResponse = await client.UploadFileAsync(new UploadFileRequest
{ FileContent = imageContent, FileName = "/" + directoryName + "/" + filename, ContentType = MimeMapping.MimeUtility.GetMimeMapping(ms.ToString()) });
}
else
{
uploadResponse = await client.UploadFileAsync(new UploadFileRequest
{ FileContent = ms.ToArray(), FileName = "/" + directoryName + "/" + filename, ContentType = MimeMapping.MimeUtility.GetMimeMapping(ms.ToString()) });
}
if (!uploadResponse.IsError)
{
finishedProcess++;
fileCounter++;
siteProcess.FinishedProcess = bucketName + " => Site is moving to MinIO...";
siteProcess.FinishedCount = finishedProcess + "/" + totalFileFolder.ToString();
siteProcess.MadePercentage = Convert.ToInt32((100 * (finishedProcess) / totalFileFolder));
}
else
{
isError = true;
}
if (finishedProcess > totalFileFolder)
{
finishedProcess = dataGridSites.SelectedItems.Count;
}
}
});
}
}
}
return isError;
}
Other function is:
private async Task<byte[]> DownloadAndCompareImages(string domain, string imgFilePath, string imgQuality)
{
using var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Add("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36");
Uri uriNormalPattern = new Uri($"https://{domain}/{imgFilePath}");
var uriWithoutQuery = uriNormalPattern.GetLeftPart(UriPartial.Path); //
var fname = System.IO.Path.GetFileName(uriWithoutQuery);
var imageBytesNormal = await httpClient.GetByteArrayAsync(uriNormalPattern); //
int imageWidth = GetImgWidth(imageBytesNormal);
var imageBytesCloudFlare = await GetCloudFlareImg(domain, imageWidth, imgQuality, imgFilePath);
if (imageBytesNormal.Length > imageBytesCloudFlare.Length)
{
return imageBytesCloudFlare;
}
else
{
return imageBytesNormal;
}
}
When using Parallel.ForEach
a lot of requests are sent to the server in a short period of times. This might overwhelm the server, but also HttpClient
is limited when accessing the same domain in parallel.
In order to mitigate this, the following steps might help:
HttpClient
instance directly, you should use IHttpClientFactory
to create the instances. For details, see this link . This avoids "resource exhaustion problems by pooling HttpMessageHandler instances".
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.