[英]upload large files (> 1 GB) to azure blob storage through web api
we have an application(.Net core) that is hosted in azure app service and we are trying to upload large files to Azure blob through web API using Form data from UI. we have an application(.Net core) that is hosted in azure app service and we are trying to upload large files to Azure blob through web API using Form data from UI. We have changed request length and API request timeout still we are facing connection time out errors even while uploading 200MB files
我们已经更改了请求长度和 API 请求超时,即使在上传 200MB 文件时我们仍然面临连接超时错误
below is the sample code I am using下面是我正在使用的示例代码
[HttpPost]
[Route("upload")]
[Consumes("multipart/form-data")]
[RequestFormLimits(MultipartBodyLengthLimit = 2147483648)]
public async Task<IHttpActionResult> Upload([FromForm] FileRequestObject fileRequestObject)
{
var url = "upload_url_to_blob_storage";
var file = fileRequestObject.Files[0];
var blob = new CloudBlockBlob(new Uri(url));
blob.Properties.ContentType = file.ContentType;
await blob.UploadFromStreamAsync(file.InputStream);
//some other operations based on file upload
return Ok();
}
public class FileRequestObject
{
public List<IFormFile> Files { get; set; }
public string JSON { get; set; }
public string BlobUrls { get; set; }
}
According to your code, you want to upload a large file to Azure blob storage as blockblob.根据您的代码,您想将一个大文件作为块块上传到 Azure blob 存储。 Please note that it has a limitation.
请注意,它有一个限制。 For more details, please refer to the document
更多详细信息,请参阅文档
The maximum size for a block blob created via Put Blob is 256 MB for version 2016-05-31 and later, and 64 MB for older versions.
对于版本 2016-05-31 及更高版本,通过 Put Blob 创建的块 Blob 的最大大小为 256 MB,旧版本为 64 MB。 If your blob is larger than 256 MB for version 2016-05-31 and later, or 64 MB for older versions, you must upload it as a set of blocks
如果您的 blob 对于版本 2016-05-31 及更高版本大于 256 MB,或者对于旧版本大于 64 MB,则必须将其作为一组块上传
So If you want to large files to azure block blob, pleae use the following steps:所以如果你想大文件到 azure 块 blob,请使用以下步骤:
For example:例如:
[HttpPost]
[Consumes("multipart/form-data")]
[RequestFormLimits(MultipartBodyLengthLimit = 2147483648)]
public async Task<ActionResult> PostAsync([FromForm]FileRequestObject fileRequestObject)
{
string storageAccountConnectionString = "DefaultEndpointsProtocol=https;AccountName=blobstorage0516;AccountKey=UVOOBCxQpr5BVueU+scUeVG/61CZbZmj9ymouAR9609WbqJhhma2N+WL/hvaoNs4p4DJobmT0F0KAs0hdtPcqA==;EndpointSuffix=core.windows.net";
CloudStorageAccount StorageAccount = CloudStorageAccount.Parse(storageAccountConnectionString);
CloudBlobClient BlobClient = StorageAccount.CreateCloudBlobClient();
CloudBlobContainer Container = BlobClient.GetContainerReference("test");
await Container.CreateIfNotExistsAsync();
CloudBlockBlob blob = Container.GetBlockBlobReference(fileRequestObject.File.FileName);
HashSet<string> blocklist = new HashSet<string>();
var file = fileRequestObject.File;
const int pageSizeInBytes = 10485760;
long prevLastByte = 0;
long bytesRemain = file.Length;
byte[] bytes;
using (MemoryStream ms = new MemoryStream())
{
var fileStream = file.OpenReadStream();
await fileStream.CopyToAsync(ms);
bytes = ms.ToArray();
}
// Upload each piece
do
{
long bytesToCopy = Math.Min(bytesRemain, pageSizeInBytes);
byte[] bytesToSend = new byte[bytesToCopy];
Array.Copy(bytes, prevLastByte, bytesToSend, 0, bytesToCopy);
prevLastByte += bytesToCopy;
bytesRemain -= bytesToCopy;
//create blockId
string blockId = Guid.NewGuid().ToString();
string base64BlockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockId));
await blob.PutBlockAsync(
base64BlockId,
new MemoryStream(bytesToSend, true),
null
);
blocklist.Add(base64BlockId);
} while (bytesRemain > 0);
//post blocklist
await blob.PutBlockListAsync(blocklist);
return Ok();
// For more information on protecting this API from Cross Site Request Forgery (CSRF) attacks, see https://go.microsoft.com/fwlink/?LinkID=717803
}
public class FileRequestObject
{
public IFormFile File { get; set; }
}
For more details, please refer to https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/
更多详情请参考https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.