Kafka 配置 PKCS12 `ssl.keystore.location=user.p12` 无需访问本地文件系统

Question

I can successfully connect to an SSL secured Kafka cluster with the following client properties:

security.protocol=SSL
ssl.truststore.type=PKCS12
ssl.truststore.location=ca.p12
ssl.truststore.password=<redacted>
ssl.keystore.type=PKCS12
ssl.keystore.location=user.p12
ssl.keystore.password=<redacted>

However, I'm writing a Java app that is running in a managed cloud environment, where I don't have access to the file system. So I can't just give it a local file path to.p12 files.

Are there any other alternatives, like using loading from S3, or from memory, or from a JVM classpath resource?

Specifically, this is a Flink app running on Amazon's Kinesis Analytics Managed Flink cluster service.

Answer 1

Sure, you can download whatever you want from wherever before you give a properties object to a KafkaConsumer. However, the user running the Java process will need some access to the local filesystem in order to download files.

I think packaging the files as part of your application JAR makes more sense, however, I don't know an easy way to refer to a classpath resource as if it were a regular filesystem path. If the code runs in YARN cluster, you can try using yarn.provided.lib.dirs option when submitting as well

Answer 2

I used a workaround tempoarily, upload your certificates to a fileshare and make your application, during initialization, dowload the certificates from the file share and save it to the location of choice like /home/site/ca.p12 then kakfa properties should read... ssl.truststore.location=/home/site/ca.p12...

Here are few lines of code to help you download and save your certificate.

// Create the Azure Files client.
    CloudFileClient fileClient = storageAccount.createCloudFileClient();
// Get a reference to the file share
    CloudFileShare share = fileClient.getShareReference("[SHARENAME]");
// Get a reference to the root directory for the share.
    CloudFileDirectory rootDir = share.getRootDirectoryReference();
// Get a reference to the directory where the file to be deleted is in
    CloudFileDirectory containerDir = rootDir.getDirectoryReference("[DIRECTORY]");
    CloudFile file = containerDir.getFileReference("[FILENAME]");
file.downloadToFile("/home/site/ca.p12");