[英]How to upload the Stream from an HttpContent result to Azure File Storage
I am attempting to download a list of files from urls stored in my database, and then upload them to my Azure FileStorage account. 我试图从存储在我的数据库中的URL下载文件列表,然后将它们上传到我的Azure FileStorage帐户。 I am successfully downloading the files and can turn them into files on my local storage or convert them to text and upload them. 我成功下载了文件,可以将它们转换为本地存储上的文件或将其转换为文本并上传。 However I lose data when converting something like a pdf to a text and I do not want to have to store the files on the Azure app that this endpoint is hosted on as I do not need to manipulate the files in any way. 但是,在将类似pdf的内容转换为文本时,我会丢失数据,而且我不希望将文件存储在此端点所托管的Azure应用程序上,因为我不需要以任何方式操作文件。
I have attempted to upload the files from the Stream
I get from the HttpContent
object using the UploadFromStream
method on the CloudFile
. 我试图使用UploadFromStream
上的CloudFile
方法从HttpContent
对象上传Stream
的文件。 Whenever this command is run I get an InvalidOperationException
with the message "Operation is not valid due to the current state of the object."
每当运行此命令时,我都会收到InvalidOperationException
,并显示消息"Operation is not valid due to the current state of the object."
I've tried converting the original Stream
to a MemoryStream
as well but this just writes a blank file to the FileStorage account, even if I set the position to the beginning of the MemoryStream
. 我也尝试将原始Stream
转换为MemoryStream
,但这只是将一个空白文件写入FileStorage帐户,即使我将位置设置为MemoryStream
的开头。 My code is below and if anyone could point out what information I am missing to make this work I would appreciate it. 我的代码在下面,如果有人能指出我缺少哪些信息来完成这项工作,我将不胜感激。
public DownloadFileResponse DownloadFile(FileLink fileLink)
{
string fileName = string.Format("{0}{1}{2}", fileLink.ExpectedFileName, ".", fileLink.ExpectedFileType);
HttpStatusCode status;
string hash = "";
using (var client = new HttpClient())
{
client.Timeout = TimeSpan.FromSeconds(10); // candidate for .config setting
client.DefaultRequestHeaders.Add("User-Agent", USER_AGENT);
var request = new HttpRequestMessage(HttpMethod.Get, fileLink.ExpectedURL);
var sendTask = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var response = sendTask.Result; // not ensuring success here, going to handle error codes without exceptions
status = response.StatusCode;
if (status == HttpStatusCode.OK)
{
var httpStream = response.Content.ReadAsStreamAsync().Result;
fileStorage.WriteFile(fileLink.ExpectedFileType, fileName, httpStream);
hash = HashGenerator.GetMD5HashFromStream(httpStream);
}
}
return new DownloadFileResponse(status, fileName, hash);
}
public void WriteFile(string targetDirectory, string targetFilePath, Stream fileStream)
{
var options = SetOptions();
var newFile = GetTargetCloudFile(targetDirectory, targetFilePath);
newFile.UploadFromStream(fileStream, options: options);
}
public FileRequestOptions SetOptions()
{
FileRequestOptions options = new FileRequestOptions();
options.ServerTimeout = TimeSpan.FromSeconds(10);
options.RetryPolicy = new NoRetry();
return options;
}
public CloudFile GetTargetCloudFile(string targetDirectory, string targetFilePath)
{
if (!shareConnector.share.Exists())
{
throw new Exception("Cannot access Azure File Storage share");
}
CloudFileDirectory rootDirectory = shareConnector.share.GetRootDirectoryReference();
CloudFileDirectory directory = rootDirectory.GetDirectoryReference(targetDirectory);
if (!directory.Exists())
{
throw new Exception("Target Directory does not exist");
}
CloudFile newFile = directory.GetFileReference(targetFilePath);
return newFile;
}
Had the same problem, the only way it worked is by reading the coming stream (in your case it is httpStream
in DownloadFile(FileLink fileLink)
method) to a byte array and using UploadFromByteArray (byte[] buffer, int index, int count)
instead of UploadFromStream
有同样的问题,只是它的工作方式是通过读取来流(在您的情况下,它是httpStream
在DownloadFile(FileLink fileLink)
法),以一个字节数组,并使用UploadFromByteArray (byte[] buffer, int index, int count)
而不是UploadFromStream
So your WriteFile(FileLink fileLink)
method will look like: 因此,您的WriteFile(FileLink fileLink)
方法将如下所示:
public void WriteFile(string targetDirectory, string targetFilePath, Stream fileStream)
{
var options = SetOptions();
var newFile = GetTargetCloudFile(targetDirectory, targetFilePath);
const int bufferLength= 600;
byte[] buffer = new byte[bufferLength];
// Buffer to read from stram This size is just an example
List<byte> byteArrayFile = new List<byte>(); // all your file will be here
int count = 0;
try
{
while ((count = fileStream.Read(buffer, 0, bufferLength)) > 0)
{
byteArrayFile.AddRange(buffer);
}
fileStream.Close();
}
catch (Exception ex)
{
throw; // you need to change this
}
file.UploadFromByteArray(allFile.ToArray(), 0, byteArrayFile.Count);
// Not sure about byteArrayFile.Count.. it should work
}
According to your description and codes, I suggest you could use Steam.CopyTo to copy the stream to the local memoryStream firstly, then upload the MemoryStream to azure file storage. 根据您的描述和代码,我建议您首先使用Steam.CopyTo将流复制到本地memoryStream,然后将MemoryStream上传到azure文件存储。
More details, you could refer to below codes: 更多细节,您可以参考以下代码:
I just change the DownloadFile method to test it. 我只是更改了DownloadFile方法来测试它。
HttpStatusCode status;
using (var client = new HttpClient())
{
client.Timeout = TimeSpan.FromSeconds(10); // candidate for .config setting
// client.DefaultRequestHeaders.Add("User-Agent", USER_AGENT);
//here I use my blob file to test it
var request = new HttpRequestMessage(HttpMethod.Get, "https://xxxxxxxxxx.blob.core.windows.net/media/secondblobtest-eypt.txt");
var sendTask = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var response = sendTask.Result; // not ensuring success here, going to handle error codes without exceptions
status = response.StatusCode;
if (status == HttpStatusCode.OK)
{
MemoryStream ms = new MemoryStream();
var httpStream = response.Content.ReadAsStreamAsync().Result;
httpStream.CopyTo(ms);
ms.Position = 0;
WriteFile("aaa", "testaa", ms);
// hash = HashGenerator.GetMD5HashFromStream(httpStream);
}
}
I had a similar problem and got to find out that the UploadFromStream
method only works with buffered streams. 我遇到了类似的问题,并且发现UploadFromStream
方法仅适用于缓冲流。 Nevertheless I was able to successfully upload files to azure storage by using a MemoryStream
. 然而,我能够通过使用MemoryStream
成功地将文件上传到azure存储。 I don't think this to be a very good solution as you are using up your memory resources by copying the content of the file stream to memory before handing it to the azure stream. 我不认为这是一个非常好的解决方案,因为在将文件流的内容复制到内存之前将文件流的内容复制到内存资源,然后将其用于内存资源。 What I have come up with is a way of writing directly to an azure stream by using instead the OpenWriteAsync
method to create the stream and then a simple CopyToAsync
from the source stream. 我想到的是一种直接写入azure流的方法,而是使用OpenWriteAsync
方法创建流,然后使用CopyToAsync
的简单CopyToAsync
。
CloudStorageAccount storageAccount = CloudStorageAccount.Parse( "YourAzureStorageConnectionString" );
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
CloudFileShare share = fileClient.GetShareReference( "YourShareName" );
CloudFileDirectory root = share.GetRootDirectoryReference();
CloudFile file = root.GetFileReference( "TheFileName" );
using (CloudFileStream fileWriteStream = await file.OpenWriteAsync( fileMetadata.FileSize, new AccessCondition(),
new FileRequestOptions { StoreFileContentMD5 = true },
new OperationContext() ))
{
await fileContent.CopyToAsync( fileWriteStream, 128 * 1024 );
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.