[英]How do you compress a stream and upload it to Azure Blob Storage without writing to a file?
I have written a Post
method in ASP.NET Core to compress the requests body and upload it to Azure Blob Storage.我在 ASP.NET Core 中编写了一个
Post
方法来压缩请求正文并将其上传到 Azure Blob 存储。 The method takes parameters as follows:该方法采用以下参数:
public async Task<IActionResult> Post([FromHeader] string AssignmentId)
Various strings are then set, including fetching the connection string for the storage:然后设置各种字符串,包括获取存储的连接字符串:
string fileName = $"{AssignmentId}.gz";
string compressedFilePath = Path.Combine(hostEnvironment.ContentRootPath, $"Test JSONs/{fileName}");
string connectionString = Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING");
I initialize the BlobClient
:我初始化
BlobClient
:
BlobClient blobClient = new BlobClient(connectionString, "assignments", fileName);
Then I create a file, and compress the body stream of the request using GZipStream
to the file:然后我创建一个文件,并使用
GZipStream
将请求的主体 stream 压缩到文件中:
using (FileStream compressedFileStream = System.IO.File.Create(compressedFilePath))
{
using GZipStream compressionStream = new GZipStream(compressedFileStream, CompressionMode.Compress);
using Stream bodyStream = HttpContext.Request.Body;
await bodyStream.CopyToAsync(compressionStream);
}
Finally I read the file I just wrote and upload using the FileStream
:最后,我阅读了我刚刚编写并使用
FileStream
上传的文件:
using (FileStream fileStream = System.IO.File.OpenRead(compressedFilePath))
{
await blobClient.UploadAsync(fileStream);
}
This solution works, but I am concerned about the constant reading and writing of the file, in terms of speed.这个解决方案有效,但我担心文件的不断读写,就速度而言。 I attempted to use a
MemoryStream
passed into the GZipStream
, however it ended up only uploading 10B files when the files should be 1KB+.我尝试使用传递给
GZipStream
的MemoryStream
,但是当文件应该为 1KB+ 时,它最终只上传了 10B 文件。
I appreciate any suggestions.我很感激任何建议。
Here is the complete method:这是完整的方法:
public async Task<IActionResult> Post([FromHeader] string AssignmentId)
{
string fileName = $"{AssignmentId}.gz";
string compressedFilePath = Path.Combine(hostEnvironment.ContentRootPath, $"Test JSONs/{fileName}");
string connectionString = Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING");
BlobClient blobClient = new BlobClient(connectionString, "assignments", fileName);
using (FileStream compressedFileStream = System.IO.File.Create(compressedFilePath))
{
using GZipStream compressionStream = new GZipStream(compressedFileStream, CompressionMode.Compress);
using Stream bodyStream = HttpContext.Request.Body;
await bodyStream.CopyToAsync(compressionStream);
}
using (FileStream fileStream = System.IO.File.OpenRead(compressedFilePath))
{
await blobClient.UploadAsync(fileStream);
}
return Ok();
}
I eventually solved this by both leaving the compression stream open, and by resetting the position of the memory stream the compression stream is writing to (thanks to @MitchWheat.). I eventually solved this by both leaving the compression stream open, and by resetting the position of the memory stream the compression stream is writing to (thanks to @MitchWheat.).
using MemoryStream memoryStream = new MemoryStream() ;
using (Stream bodyStream = HttpContext.Request.Body)
{
using (GZipStream compressionStream = new GZipStream(memoryStream,
CompressionMode.Compress, true))
{
await bodyStream.CopyToAsync(compressionStream);
}
}
memoryStream.Position = 0;
await blobClient.UploadAsync(memoryStream, overwrite: true);
The Azure Storage library offers a push-based stream to upload content that works well with compression. Azure 存储库提供了一个基于推送的 stream 来上传适用于压缩的内容。
const int PageSize = 1024 * 1024; // 1 MB
using var sourceFile = new FileStream(file.FullName, FileMode.Open, FileAccess.Read, FileShare.Read);
using var destinationStream = await blobClient.OpenWriteAsync(overwrite: true, new() { BufferSize = PageSize }, cancellationToken);
using var compressedContent = new GZipStream(destinationStream, CompressionMode.Compress, leaveOpen: true);
await sourceFile.CopyToAsync(compressedContent, cancellationToken);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.