Python Confluent Kafka Client in Docker

Question

When setting up my Kafka Client locally on my mac without specifying ssl.ca.location, it works. But when I run it on a docker container locally I get the error: Failed to verify broker certificate: unable to get local issuer certificate . How can I solve this?

driver_options = {
    'bootstrap.servers': kafka_brokers_sasl,
    'sasl.mechanisms': 'PLAIN',
    'security.protocol': 'SASL_SSL',
     'sasl.username': 'token',
     'sasl.password': api_key,
     'log.connection.close' : True,
     #'ssl.ca.location': ''
}

I have seen multiple proposals for trying this, for example this solution: https://github.com/henadzit/cloudkarafka-test-project , but can't get it to work.

Answer 1

Failed to verify broker certificate: unable to get local issuer certificate

Indicates that librdkafka is not able to find the root CA certificate on the server. Normally you have two options:

1. You either let librdkafka locate the certificate on the host
2. or you can specify the location of the certificate

Docker images though, are usually unable to locate the CA certificate so you can either install it on the image or specify its location in librdkafka 's configuration

To do so, you need to specify ca-cert's location in ssl.ca.location

EDIT:
In order to create certificates you can follow Confluent's Security tutorial which is a step-by-step guide.

In order to encrypt with SSL on Kubernetes you can use kube-lego