Kafka 使用 TLS 连接 mongoDB 接收器

Question

I set up my mongoDB cluster with TLS authentication.

I can successfully connect on a mongos instance using :

/opt/cluster/stacks/mongoDB/bin/mongosh --tls --host $(hostname).domain.name -tlsCAFile /opt/cluster/security/ssl/cert.pem --port 27017

I have a Kafka connect mongoDB sink that has the following configuration :

{
  "name": "client-order-request-mongodb-sink",
  "config": {
    "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector",
    "database":"Trading",
    "collection":"ClientOrderRequest",
    "topics":"ClientOrderRequest",
    "connection.uri":"mongodb://hostname1.domain.name:27017,pre-hostname2.domain.name:27017",
    "mongo.errors.tolerance": "all",
    "mongo.errors.log.enable": "true",
    "errors.log.include.messages": "true",
    "writemodel.strategy":"com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy",
    "document.id.strategy": "com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy",
    "document.id.strategy.overwrite.existing": "true",
    "document.id.strategy.partial.value.projection.type": "allowlist",
    "document.id.strategy.partial.value.projection.list": "localReceiveTime,clientId,orderId"
  }
}

It is working fine if I redeploy mongoDB without authentication, but now when I try to instantiate it with the following curl command :

curl -X POST -H "Content-Type: application/json" --data '@connect-task-sink-mongodb-client-order-request.json' $KAFKA_CONNECT_LEADER_NODE/connectors/

I have the following error:

{"error_code":400,"message":"Connector configuration is invalid and contains the following 1 error(s):\nUnable to connect to the server.\nYou can also find the above list of errors at the endpoint /connector-plugins/{connectorType}/config/validate "}

From the mongoDB kafka connect sink documentation I found that I needed to set up global variable of the KAFKA_OPTS so before starting the distributed connect server I do:

export KAFKA_OPTS="\
-Djavax.net.ssl.trustStore=/opt/cluster/security/ssl/keystore.jks \
-Djavax.net.ssl.trustStorePassword=\"\" \
-Djavax.net.ssl.keyStore=/opt/cluster/security/ssl/keystore.jks \
-Djavax.net.ssl.keyStorePassword=\"\""

Notice that I put an empty password because when I list the entry of my keystore with:

keytool -v -list -keystore key.jks

Then I just press enter when the password is prompted.

Answer 1

So the issue was that the ssl connection wasn't enabled on the client side.

If you want to do so with the mongoDB kafka connect plugin you need to state it in the connection.uri config parameter such as:

"connection.uri":"mongodb://hostname1.domain.name:27017,pre-hostname2.domain.name:27017/?ssl=true"