[英]Kafka SSL handshake failed in custom Java producer
Trying to produce some data using my Kafka producer application, but i get below error:尝试使用我的 Kafka 生产者应用程序生成一些数据,但出现以下错误:
[SocketServer brokerId=0] Failed authentication with localhost/127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[SocketServer brokerId=0] 与 localhost/127.0.0.1 的身份验证失败(SSL 握手失败)(org.apache.kafka.common.network.Selector)
I use SASL_SSL protocol with PLAIN mechanism to communicate with Kafka.我使用带有 PLAIN 机制的 SASL_SSL 协议与 Kafka 进行通信。 When I use kafka-console-producer
sh kafka-console-producer.sh --broker-list localhost:9093 --topic kafka-topic --producer.config../config/producer.properties
当我使用 kafka-console-producer
sh kafka-console-producer.sh --broker-list localhost:9093 --topic kafka-topic --producer.config../config/producer.properties
and kafka-console-consumer sh kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic kafka-topic --consumer.config../config/consumer.properties
和 kafka-console-consumer
sh kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic kafka-topic --consumer.config../config/consumer.properties
everything works fine.一切正常。 Here's a part of my server.properties :
这是我的server.properties的一部分:
listeners=PLAINTEXT://localhost:9092,SASL_SSL://localhost:9093
advertised.listeners=PLAINTEXT://localhost:9092,SASL_SSL://localhost:9093
listener.name.sasl_ssl.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="admin-secret" \
user_admin="admin-secret";
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
ssl.keystore.location=/mnt/data/kafka/config/keystore/kafka.keystore.jks
ssl.keystore.password=password
ssl.key.password=password
ssl.truststore.location=/mnt/data/kafka/config/truststore/kafka.truststore.jks
ssl.truststore.password=password
ssl.client.auth=required
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.keystore.type=JKS
ssl.truststore.type=JKS
ssl.secure.random.implementation=SHA1PRNG
producer.properties生产者属性
bootstrap.servers=localhost:9093
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="admin-secret" \
user_admin="admin-secret";
ssl.truststore.location=/mnt/data/kafka/config/truststore/kafka.truststore.jks
ssl.truststore.password=password
consumer.properties消费者属性
bootstrap.servers=localhost:9093
group.id=test-consumer-group
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="admin-secret" \
user_admin="admin-secret";
ssl.truststore.location=/mnt/data/kafka/config/truststore/kafka.truststore.jks
ssl.truststore.password=password
And here's my Java Kafka producer application这是我的 Java Kafka 生产者应用程序
private KafkaProducer<String, String> producer;
private String address;
private final int BATCH_SIZE = 16384 * 4;
private Properties setProperties() {
Properties properties = new Properties();
properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, address);
properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
properties.put(ProducerConfig.BATCH_SIZE_CONFIG, BATCH_SIZE);
properties.put(ProducerConfig.LINGER_MS_CONFIG, 200);
properties.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "snappy");
properties.put("acks", "all");
properties.put("sasl.mechanism", "PLAIN");
properties.put("security.protocol", "SASL_SSL");
properties.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-secret\" user_admin=\"admin-secret\";");
properties.put("ssl.truststore.location", "/mnt/data/kafka/config/truststore/kafka.truststore.jks");
properties.put("ssl.truststore.password", "password");
return properties;
}
public void createTopicWithPartitions(String topicName, int partitionsCount) throws ExecutionException, InterruptedException {
Properties properties = new Properties();
properties.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, address);
AdminClient adminClient = AdminClient.create(properties);
boolean isTopicExists = adminClient.listTopics().names().get().stream()
.anyMatch(name -> name.equals(topicName));
if (isTopicExists) {
System.out.println("Topic already exists");
} else {
NewTopic newTopic = new NewTopic(topicName, partitionsCount, (short) 1);
adminClient.createTopics(Collections.singleton(newTopic)).all().get();
}
adminClient.close();
}
public void sendMessages(String topicName, String payload, int messagesCount) {
for (int i = 0; i < messagesCount; i++) {
String partitionKey = DataUtils.generateSourceDeviceId(15).toUpperCase();
producer.send(new ProducerRecord<>(topicName, partitionKey, payload));
}
}
public KafkaMessagesProducer(String address) {
this.address = address;
this.producer = new KafkaProducer<>(setProperties());
}
public int getBATCH_SIZE() {
return BATCH_SIZE;
}
As I described before console producer/consumer works fine, my Java application gets SSL handshake error, but with switched off SASL_SSL protocol my Java app works fine.正如我之前描述的控制台生产者/消费者工作正常,我的 Java 应用程序得到 SSL 握手错误,但关闭 SASL_SSL 协议我的 ZD52387880E1EA22817A72D37592138 工作正常。
UPD : Certificate generation tool used from this website: https://github.com/confluentinc/confluent-platform-security-tools/blob/master/kafka-generate-ssl.sh
UPD :本网站使用的证书生成工具:
https://github.com/confluentinc/confluent-platform-security-tools/blob/master/kafka-generate-ssl.sh
I had a problem with createTopicWithPartitions method.我对 createTopicWithPartitions 方法有疑问。 I rewrote properties created in setProperties() method by (AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, address)
我重写了(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG,地址)在 setProperties() 方法中创建的属性
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.