IBM message hub communciation with C# confluent api

Question

I have been trying to find documentation to connect to IBM Message Hub through C# Apache Kafka Confluent API, but have not been successful. The official repo on github does not have sample for C#. Has anyone been able to communicate to ibm message hub using C#, if yes can you share the process.

Thanks.

Update: I have had success in communicating with the IBM Message Hub.

Libraries:

1. librdkafka -... 0.11.0-RC2

2. Certificate From : https://curl.haxx.se/docs/caextract.html

3. Confluent.kafka.dll Confluent.Kafka 0.11.0-RC1

Config:

private static Dictionary<string, object> constructConfig(string brokerList, bool enableAutoCommit) =>
            new Dictionary<string, object>
            {
                { "group.id", "history" },
                { "enable.auto.commit", enableAutoCommit },
                { "auto.commit.interval.ms", 5000 },
                { "statistics.interval.ms", 60000 },
                { "bootstrap.servers", "ibmserver:port" },
                { "default.topic.config", new Dictionary<string, object>()
                    {
                        { "auto.offset.reset", "smallest" }
                    }
                },
                {"ssl.ca.location",@"E:\cert\cacert.pem" },              
                {"api.version.request","true" },
                {"security.protocol","sasl_ssl" },
                {"sasl.mechanisms","PLAIN" },
                {"sasl.username","xxxx" },
                {"sasl.password","xxxxx" }

            };

.net Version: 4.5.2

Hope it saves time for someone.

Thanks to Edoardo Comar for guiding me to much needed information.

Answer 1

We have not written a C# sample yet.

The Confluent C# Kafka client is a wrapper around the C library librdkafka which until version 0.9.5 (the last release at the time of writing this post) could not be built for Windows with SASL_SSL support which is necessary for authenticating against Message Hub.

However that has changed with librdkafka 0.11 (in release candidate as I'm posting this comment) thanks to some very recent commits.

I verified that librdkafka 0.11 (master branch) is able to authenticate with Message Hub on Windows. You will need to set these configuration properties:

api.version.request=true
security.protocol=sasl_ssl
ssl.ca.location=<path to a valid cert.pem file>
sasl.mechanisms=PLAIN
sasl.username=<username from your Bluemix credentials>
sasl.password=<password from your Bluemix credentials>

I didn't know how to obtain a valid .pem certificate file in Windows, so I copied over a cert.pem file from macOS (installed via brew openssl in /usr/local/etc/openssl ). A cert.pem from Ubuntu (found in /etc/openssl ) should work as well.

Good luck and please keep me updated with your progress,