In C#/Mono, not able to communicate with google cloud storage

Question

I'm developing a C# app right now that runs on Linux instances via Mono. It needs to store images in Google Cloud Storage when it's done processing them.

I have the auth credentials setup and the .NET client library is able to setup the StorageService object (ie find the JSON file and validate everything). When I tried to upload, nothing happened. So in my SetupConnection() function, I tried to query the single bucket I have on my project. I got this error:

Unhandled Exception:
System.Net.WebException: Error writing request: The authentication or decryption has failed.
  at System.Net.WebConnectionStream.WriteHeaders () [0x00000] in <filename unknown>:0 
  at System.Net.WebConnectionStream.SetHeaders (System.Byte[] buffer) [0x00000] in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.Net.WebConnectionStream:SetHeaders (byte[])
  at System.Net.HttpWebRequest.SendRequestHeaders (Boolean propagate_error) [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Http.ConfigurableMessageHandler+<SendAsync>d__43.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions+<ExecuteAsync>d__0.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[Google.Apis.Auth.OAuth2.Responses.TokenResponse].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceAccountCredential+<RequestAccessTokenAsync>d__19.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Boolean].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceCredential+<GetAccessTokenForRequestAsync>d__23.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.String].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceAccountCredential+<GetAccessTokenForRequestAsync>d__20.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.String].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceCredential+<InterceptAsync>d__21.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Http.ConfigurableMessageHandler+<SendAsync>d__43.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Requests.ClientServiceRequest`1+<ExecuteUnparsedAsync>d__26[Google.Apis.Storage.v1.Data.Buckets].MoveNext () [0x00000] in <filename unknown>:0 
[ERROR] FATAL UNHANDLED EXCEPTION: System.Net.WebException: Error writing request: The authentication or decryption has failed.
  at System.Net.WebConnectionStream.WriteHeaders () [0x00000] in <filename unknown>:0 
  at System.Net.WebConnectionStream.SetHeaders (System.Byte[] buffer) [0x00000] in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.Net.WebConnectionStream:SetHeaders (byte[])
  at System.Net.HttpWebRequest.SendRequestHeaders (Boolean propagate_error) [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Http.ConfigurableMessageHandler+<SendAsync>d__43.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions+<ExecuteAsync>d__0.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[Google.Apis.Auth.OAuth2.Responses.TokenResponse].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceAccountCredential+<RequestAccessTokenAsync>d__19.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Boolean].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceCredential+<GetAccessTokenForRequestAsync>d__23.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.String].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceAccountCredential+<GetAccessTokenForRequestAsync>d__20.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.String].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Auth.OAuth2.ServiceCredential+<InterceptAsync>d__21.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Http.ConfigurableMessageHandler+<SendAsync>d__43.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[System.Net.Http.HttpResponseMessage].GetResult () [0x00000] in <filename unknown>:0 
  at Google.Apis.Requests.ClientServiceRequest`1+<ExecuteUnparsedAsync>d__26[Google.Apis.Storage.v1.Data.Buckets].MoveNext () [0x00000] in <filename unknown>:0 

Here's my setup code:

public static void SetupGCSConnection()
{
    var credentials = Google.Apis.Auth.OAuth2.GoogleCredential.GetApplicationDefaultAsync().Result;

    if (credentials.IsCreateScopedRequired)
        credentials = credentials.CreateScoped(new[] { StorageService.Scope.DevstorageFullControl });

    var serviceInitializer = new BaseClientService.Initializer()
    {
        ApplicationName = "MyApp",
        HttpClientInitializer = credentials
    };

    storage = new StorageService(serviceInitializer);

    // Error is thrown here
    var buckets = storage.Buckets.List(Config.GoogleProjectID).Execute();
    foreach (var b in buckets.Items) {
        Console.WriteLine(b.ToString());
    }
}

I've been able to get C# to talk to GCS before, but that was when I was working in a Windows environment, with Microsoft's .NET vm. I did a little bit of research prior and there might be some issues with how Mono handles CAs versus Microsoft's VM does.

Answer 1

Your problem usually comes from the root certificates not being updated.

Follow the instructions here to update your root certificates.

If that doesn't helps then you can set a delegate to

ServicePointManager.ServerCertificateValidationCallback

and accept all the certificates. If that works then you know the problem is on the certificate validation and you can try another solutions. (I would NEVER recommend to use the callback to accept all the certificates in production, just use it to debug your problem).

Answer 2

I fixed the problem.

I was using the default Mono version that came with Ubuntu, and it was out of date. I switched to using the official Mono repositories here: http://www.mono-project.com/docs/getting-started/install/linux/

And then installed the ca-certificates-mono package as well. (More details here: http://www.mono-project.com/docs/faq/security/ )

Thanks to Lex Li for helping.