Access self signed X509certificates in XamarinForms for mqtt TLS connection to a Mosquitto broker

Question

I desire to TLS secure with a self signed x509certificate a number of existing XamarinForms apps that connect to a mosquitto mqtt broker using the M2MqttDotnetCore client.

To that end I have created a simple sample XamarinForms pub/sub chat app to learn how to secure an XamarinForms mqtt client application that can be sound in this GitHub repository. jhalbrecht/XamarinFormsMqttSample

I have samples in Mosquitto_pub, python and a .net console app that accomplish this goal of successfully connecting to a mosquitto broker over port 8883 with TLS and a self signed certificate. The XamarinForms UWP app also works unsecured and secured. I'm having trouble getting the Android app to work with TLS on port 8883 , The Android app does work unsecured on port 1883. This is the runtime log from Visual Studio 2017

[0:] M2Mqtt.Exceptions.MqttConnectionException: Exception connecting to the broker ---> System.AggregateException: One or more errors occurred. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
  at /Users/builder/jenkins/workspace/xamarin-android-d15-9/xamarin-android/external/mono/external/boringssl/ssl/handshake_client.c:1132
  at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00038] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status) [0x0003e] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus)
  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
  at Mono.Net.Security.AsyncProtocolRequest+<ProcessOperation>d__24.MoveNext () [0x000ff] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.AsyncProtocolRequest+<StartOperation>d__23.MoveNext () [0x0008b] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
   --- End of inner exception stack trace ---
  at Mono.Net.Security.MobileAuthenticatedStream+<ProcessAuthentication>d__47.MoveNext () [0x00254] in <fb6d78e506844b3b96d5b35aa047fbbd>:0 
   --- End of inner exception stack trace ---
  at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <d4a23bbd2f544c30a48c44dd622ce09f>:0 
  at System.Threading.Tasks.Task.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00043] in <d4a23bbd2f544c30a48c44dd622ce09f>:0 
  at System.Threading.Tasks.Task.Wait () [0x00000] in <d4a23bbd2f544c30a48c44dd622ce09f>:0 
  at M2Mqtt.Net.MqttNetworkChannel.Connect () [0x000a8] in <72fbe921f857483bafbb8b397ec98dd1>:0 
  at M2Mqtt.MqttClient.Connect (System.String clientId, System.String username, System.String password, System.Boolean willRetain, System.Byte willQosLevel, System.Boolean willFlag, System.String willTopic, System.String willMessage, System.Boolean cleanSession, System.UInt16 keepAlivePeriod) [0x0001e] in <72fbe921f857483bafbb8b397ec98dd1>:0 
   --- End of inner exception stack trace ---
  at M2Mqtt.MqttClient.Connect (System.String clientId, System.String username, System.String password, System.Boolean willRetain, System.Byte willQosLevel, System.Boolean willFlag, System.String willTopic, System.String willMessage, System.Boolean cleanSession, System.UInt16 keepAlivePeriod) [0x00037] in <72fbe921f857483bafbb8b397ec98dd1>:0 
  at M2Mqtt.MqttClient.Connect (System.String clientId) [0x00000] in <72fbe921f857483bafbb8b397ec98dd1>:0 
  at MqttDataServices.Services.MqttDataService+<Initialize>d__5.MoveNext () [0x00266] in C:\jstuff\MqttSample\MqttDataServices\Services\MqttDataService.cs:183 

The way I am currently loading and accessing the X509certificates is not secure or a best practice. It works. I hope to eventually learn how to access the device ca keystores for each mobile platform. I use the cross-platform plug-in FilePicker to load a cert, base64 encode it, and save it.

 FileData fileData = await Plugin.FilePicker.CrossFilePicker.Current.PickFile();
 if (fileData == null)
 return; // user canceled file picking

 string fileName = fileData.FileName;
 string content = Convert.ToBase64String(fileData.DataArray, 0, fileData.DataArray.Length,
 Base64FormattingOptions.None);

 string deviceFileName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), fileName);
 File.WriteAllText(deviceFileName, content);      

I have reached out to a few Xamarin folks via twitter. I have an open issue in my above mentioned repository discussing the problem where @baulig from Microsoft has I believe given me the answer however I don't currently know how to implement it.

I just looked at the certificate validation code and what it does is essentially

var certStore = KeyStore.GetInstance ("AndroidCAStore"); certStore.Load(null);

This is the entry point: https://github.com/mono/mono/blob/master/mcs/class/System/Mono.Btls/MonoBtlsX509LookupAndroid.cs , it calls this code https://github.com/mono/mono/blob/master/mcs/class/System/System/AndroidPlatform.cs#L101 which then calls into xamarin-android code here: https://github.com/xamarin/xamarin-android/blob/master/src/Mono.Android/Android.Runtime/AndroidEnvironment.cs

The KeyStore should be this class: https://developer.xamarin.com/api/type/Java.Security.KeyStore/ .

So you should be able to do this via Java.Security.KeyStore .

• What permissions are necessary to grant in AndroidManifest.xml?
• What terms might I research to properly access the platform ca keystores?

Additions after initial posting

• February 27, 2019 (MST) 2:51 PM
  Added certs and mqtt client creation from MqttDataService.cs

 X509Certificate caCert = X509Certificate.CreateFromCertFile(Path.Combine(filesDirectoryBasePath, "ca.crt"));
 string thePfxPathOnDevice = Path.Combine(filesDirectoryBasePath, "xamarinclient.pfx");
 string theBase64EncodedPfx = File.ReadAllText(thePfxPathOnDevice);

 byte[] certificate = Convert.FromBase64String(theBase64EncodedPfx);
 X509Certificate2 clientCert = new X509Certificate2(certificate, "xamarin");
 _client = new MqttClient(
     GetHostName(_xpdSetting.MqttBrokerAddress),
     Int32.Parse(_xpdSetting.MqttBrokerTlsPort),
     _xpdSetting.UseTls,
     caCert,
     clientCert,
     MqttSslProtocols.TLSv1_2
     //MyRemoteCertificateValidationCallback
     );

Answer 1

Since you are using .Net's/Mono Socket (via M2MqttDotnetCore), just use cert pinning and you only have to handle the RemoteCertificateValidationCallback . Thus no messing with Android's trusted stores, etc...

SslStream Usage on Android:

Note: There are issues with SslStream on Android, object allocations can go crazy... I believe(?) there is an open issue about this. (I had to use Java's SSLSocket a couple times to work around this issue)

Enable Native TLS 1.2+

• Using BoringSSL via the Android project build options

Add your cert to the Android's Asset directory:

├── Assets
│   └── sushihangover.cert

• This is your cert/.pem file ( NOT your KEY!! )

• Make sure that this is an ascii file with no unicode BOM header

• Via openssl example (just change it to your host and secure port)

   echo -n | openssl s_client -connect 10.1.10.250:5001 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' 

RemoteCertificateValidationCallback Implementation

Note : The following code can be in used in NetStd2.0 or Xamarin.Android

X509Certificate sushihangoverCert; // Class level var

bool CertificateValidation(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors certificateErrors)
{
    if (sushihangoverCert == null)
    {
        // There is no non-async version of OpenAppPackageFileAsync (via Xamarin.Essential) 😡 Why!!!
        using (var waitHandle = new EventWaitHandle(false, EventResetMode.AutoReset))
        {
            Task.Run(async () =>
            {
                using (var assetStream = await Xamarin.Essentials.FileSystem.OpenAppPackageFileAsync("sushihangover.cert"))
                using (var memStream = new MemoryStream())
                {
                    assetStream.CopyTo(memStream);
                    sushihangoverCert = new X509Certificate(memStream.ToArray());
                    waitHandle.Set();
                }
            });
            waitHandle.WaitOne();
        }
    }
    return sushihangoverCert.Equals(certificate) ? true : false;
}

SSLStream Usage Example:

Note: This is connecting to a NetCore Web API port using a self-signed cert

using (var tcpClient = new TcpClient("10.1.10.250", 5001))
using (var ssl = new SslStream(tcpClient.GetStream(), false, new RemoteCertificateValidationCallback(CertificateValidation)))
{
    ssl.AuthenticateAsClient("10.1.10.250", null, System.Security.Authentication.SslProtocols.Tls12, false);
    if (ssl.CanWrite)
    {
        var send = Encoding.ASCII.GetBytes("GET /api/item HTTP/1.1\r\nhost: 10.1.10.250\r\n\r\n");
        await ssl.WriteAsync(send, 0, send.Length);
        var buffer = new byte[4096];
        var count = await ssl.ReadAsync(buffer, 0, buffer.Length);
        Console.WriteLine(Encoding.UTF8.GetString(buffer, 0, count));
    }
    else
        throw new SocketException();
}

Server cert mismatch error:

If your server cert (self-signed or not) does not match the one that you are pinning to, you will receive:

{Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED