C＃+ Mono：运行Azure-Storage SDK时出错

Question

I have a application that running great on Windows environment. The application written with C# 4.6.1 and using Azure Storage to upload\\download blobs.

I want to run the same application on linux ubuntu server (v16). After installing mono on the server, I tried to run the application. The program crashed with exception.

So, I added logs and found that the problem comming from Azure-Storage SDK (v7.0.0, installed via nu-get).

This is my relevant code:

CloudBlockBlob blob = container.GetBlockBlobReference(string.Join("", blob_path));
using (MemoryStream ms = new MemoryStream())
{
    blob.DownloadToStream(ms); // <--- BOOM
    //...
}

This is the exception I get when running with Mono:

Microsoft.WindowsAzure.Storage.StorageException: read failed 
---> System.IO.IOException: read failed 
---> System.InvalidOperationException: Operation is not valid due to the current state of the object.
  at Mono.Net.Security.MobileAuthenticatedStream.CheckThrow (System.Boolean authSuccessCheck) [0x0001b] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.get_Context () [0x00000] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessRead (Mono.Net.Security.AsyncProtocolRequest asyncRequest, Mono.Net.Security.AsyncOperationStatus status) [0x00011] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (Mono.Net.Security.AsyncOperationStatus status) [0x0006b] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation () [0x0000d] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Mono.Net.Security.AsyncProtocolRequest.StartOperation () [0x00000] in <2b0d86369d72459baed0cee98a8e578a>:0 
   --- End of inner exception stack trace ---
  at Mono.Net.Security.MobileAuthenticatedStream.EndReadOrWrite (System.IAsyncResult asyncResult, Mono.Net.Security.AsyncProtocolRequest& nestedRequest) [0x00055] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.EndRead (System.IAsyncResult asyncResult) [0x00000] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at System.Net.WebConnection.EndRead (System.Net.HttpWebRequest request, System.IAsyncResult result) [0x000d4] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at System.Net.WebConnectionStream.EndRead (System.IAsyncResult r) [0x000af] in <2b0d86369d72459baed0cee98a8e578a>:0 
  at Microsoft.WindowsAzure.Storage.Core.ByteCountingStream.EndRead (System.IAsyncResult asyncResult) [0x00000] in <1a6da444611441028237826045bb8126>:0 
  at Microsoft.WindowsAzure.Storage.Core.Util.StreamExtensions.WriteToSync[T] (System.IO.Stream stream, System.IO.Stream toStream, System.Nullable`1[T] copyLength, System.Nullable`1[T] maxLength, System.Boolean calculateMd5, System.Boolean syncRead, Microsoft.WindowsAzure.Storage.Core.Executor.ExecutionState`1[T] executionState, Microsoft.WindowsAzure.Storage.Core.Util.StreamDescriptor streamCopyState) [0x002c6] in <1a6da444611441028237826045bb8126>:0 
  at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T] (Microsoft.WindowsAzure.Storage.Core.Executor.RESTCommand`1[T] cmd, Microsoft.WindowsAzure.Storage.RetryPolicies.IRetryPolicy policy, Microsoft.WindowsAzure.Storage.OperationContext operationContext) [0x003c7] in <1a6da444611441028237826045bb8126>:0 
   --- End of inner exception stack trace ---
  at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T] (Microsoft.WindowsAzure.Storage.Core.Executor.RESTCommand`1[T] cmd, Microsoft.WindowsAzure.Storage.RetryPolicies.IRetryPolicy policy, Microsoft.WindowsAzure.Storage.OperationContext operationContext) [0x006b5] in <1a6da444611441028237826045bb8126>:0 
  at Microsoft.Wind… Read more                        
  at Microsoft.WindowsAzure.Storage.Core.ByteCountingStream.EndRead (System.IAsyncResult asyncResult) [0x00000] in <1a6da444611441028237826045bb8126>:0 
  at Microsoft.WindowsAzure.Storage.Core.Util.StreamExtensions.WriteToSync[T] (System.IO.Stream stream, System.IO.Stream toStream, System.Nullable`1[T] copyLength, System.Nullable`1[T] maxLength, System.Boolean calculateMd5, System.Boolean syncRead, Microsoft.WindowsAzure.Storage.Core.Executor.ExecutionState`1[T] executionState, Microsoft.WindowsAzure.Storage.Core.Util.StreamDescriptor streamCopyState) [0x002c6] in <1a6da444611441028237826045bb8126>:0
  at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T] (Microsoft.WindowsAzure.Storage.Core.Executor.RESTCommand`1[T] cmd, Microsoft.WindowsAzure.Storage.RetryPolicies.IRetryPolicy policy, Microsoft.WindowsAzure.Storage.OperationContext operationContext) [0x003c7] in <1a6da444611441028237826045bb8126>:0 
   --- End of inner exception stack trace ---
  at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T] (Microsoft.WindowsAzure.Storage.Core.Executor.RESTCommand`1[T] cmd, Microsoft.WindowsAzure.Storage.RetryPolicies.IRetryPolicy policy, Microsoft.WindowsAzure.Storage.OperationContext operationContext) [0x006b5] in <1a6da444611441028237826045bb8126>:0 
  at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.DownloadRangeToStream (System.IO.Stream target, System.Nullable`1[T] offset, System.Nullable`1[T] length, Microsoft.WindowsAzure.Storage.AccessCondition accessCondition, Microsoft.WindowsAzure.Storage.Blob.BlobRequestOptions options, Microsoft.WindowsAzure.Storage.OperationContext operationContext) [0x00033] in <1a6da444611441028237826045bb8126>:0 
  at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.DownloadToStream (System.IO.Stream target, Microsoft.WindowsAzure.Storage.AccessCondition accessCondition, Microsoft.WindowsAzure.Storage.Blob.BlobRequestOptions options, Microsoft.WindowsAzure.Storage.OperationContext operationContext) [0x00000] in <1a6da444611441028237826045bb8126>:0 
  at ***.DownloadFile (System.Uri url, System.IO.FileInfo file) [0x000e8] in <1a85ad8f477c4d6f8a3af08c9e99e976>:0 
  at ***.TimedCache+<Add>d__32.MoveNext () [0x00160] in <1a85ad8f477c4d6f8a3af08c9e99e976>:0 
Request Information
RequestID:2025b933-0001-0031-2325-10951e000000
RequestDate:Tue, 08 Aug 2017 09:08:58 GMT
StatusMessage:Partial Content

The Linux machine is running under Google-Cloud. One direction for solving this is the Goolge-Cloud firewall. Maybe the firewall blocking the communication with Azure-Storage?

My code is basically downloading a blob from the storage and save it locally. The strange thing is that when I checking the directory, where the files are being saved, I can find there some files. What's mean, it's working part of the times.

How to solve the problem?

UPDATE

According to @Amor, I tried to understand where is the problem. We have two options:

• The problem is with the C# code or Mono.
• The problem is with the environment (Linux \\ google-cloud \\ ...)

I downloaded the code called "Storage_REST_CS.zip" on this page . Update the code as needed (insert my credientials) and tested it in Windows environment - working good. I took the binaries and moved them to test it on my Linux machine. I got error:

System.Net.WebException: The remote server returned an error: (403) Server failed to authenticate the request. Make sure the value of Authorization headers formed correctly including the signature.
    at StorageSampleREST.RESTHelper.Retry[T] (StorageSampleREST.RESTHelper+RetryDelegate`1[T] del, System.Int32 numberOfRetries, System.Int32 msPause) [0x0001d] in <033ea4d1a16740f3b711001b7c2f09f5>:0 
    at StorageSampleREST.RESTHelper.Retry[T] (StorageSampleREST.RESTHelper+RetryDelegate`1[T] del) [0x00001] in <033ea4d1a16740f3b711001b7c2f09f5>:0 
    at StorageSampleREST.BlobHelper.ListContainers () [0x00001] in <033ea4d1a16740f3b711001b7c2f09f5>:0 
    at StorageSampleREST.Program.TestBlobStorage () [0x00023] in <033ea4d1a16740f3b711001b7c2f09f5>:0

Now I tried to understand where is the problem - In my code (c#) or with the environment (Linux - Google cloud). So I extracted the HTTP request that executed (with failure on Mono) and tried to run it with cURL directly:

curl -i -H "x-ms-date: Wed, 09 Aug 2017 09:03:46 GMT
x-ms-version: 2009-09-19
Authorization: SharedKey userfiles:***" -X GET https://***.blob.core.windows.net/?comp=list

And the result was 200 OK. What's mean, it's not a problem of any Firewall, Linux environment or connectivity.

Now, the focus should be on Mono settings since executing the same HTTP request with cURL finished successfully (200). It's not depends on my specific code, since similar behavior happen also with the sample code mentioned above.

Thank you.

Answer 1

Azure Storage Client SDK will communicate with the Storage Server with Https and your client side will validate the certificate received from server side. After migrate to MONO, please add following code at the top of your code to validate the certificate.

ServicePointManager.ServerCertificateValidationCallback = MyRemoteCertificateValidationCallback;

public static bool  MyRemoteCertificateValidationCallback(Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
    bool isOk = true;
    // If there are errors in the certificate chain,
    // look at each error to determine the cause.
    if (sslPolicyErrors != SslPolicyErrors.None)
    {
        for (int i = 0; i < chain.ChainStatus.Length; i++)
        {
            if (chain.ChainStatus[i].Status == X509ChainStatusFlags.RevocationStatusUnknown)
            {
                continue;
            }
            chain.ChainPolicy.RevocationFlag = X509RevocationFlag.EntireChain;
            chain.ChainPolicy.RevocationMode = X509RevocationMode.Online;
            chain.ChainPolicy.UrlRetrievalTimeout = new TimeSpan(0, 1, 0);
            chain.ChainPolicy.VerificationFlags = X509VerificationFlags.AllFlags;
            bool chainIsValid = chain.Build((X509Certificate2)certificate);
            if (!chainIsValid)
            {
                isOk = false;
                break;
            }
        }
    }
    return isOk;
}