[英]apache kafka partition,java
我对Apache Kafka非常陌生,我正在尝试将我的字符串发送给Kafka时进行Kafka分区,
public class Producer {
public static void main(String[] argv)throws Exception {
String topicName = "test";
//Configure the Producer
Properties configProperties = new Properties();
configProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"192.168.4.226:9092");
configProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.ByteArraySerializer");
configProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");
configProperties.put(ProducerConfig.PARTITIONER_CLASS_CONFIG,CountryPartitioner.class.getCanonicalName());
configProperties.put("partitions.0","USA");
configProperties.put("partitions.1","India");
org.apache.kafka.clients.producer.Producer producer = new KafkaProducer(configProperties);
ProducerRecord<String, String> rec = new ProducerRecord<String, String>(topicName,"message111");
producer.send(rec, new Callback() {
public void onCompletion(RecordMetadata metadata, Exception exception) {
System.out.println("Message sent to topic ->" + metadata.topic() +" stored at offset->" + metadata.offset());
}
});
producer.close();
}
}
现在我遇到这样的异常
2018-07-03 17:52:12 ERROR RecordBatch:102 - Error executing user-provided callback on message for topic-partition test-2:
java.lang.NullPointerException
at com.spnotes.kafka.partition.Producer$1.onCompletion(Producer.java:29)
at org.apache.kafka.clients.producer.internals.RecordBatch.done(RecordBatch.java:99)
at org.apache.kafka.clients.producer.internals.RecordBatch.maybeExpire(RecordBatch.java:136)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortExpiredBatches(RecordAccumulator.java:220)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:192)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:141)
at java.lang.Thread.run(Thread.java:748)
我做错了什么? 提前致谢。
编辑3:只有要做的就是创建多个分区,请参阅以下步骤。首先我们为每个代理创建一个配置文件(在Windows上使用copy命令)
> cp config/server.properties config/server-1.properties
> cp config/server.properties config/server-2.properties
现在编辑这些新文件并设置以下属性:
config/server-1.properties:
broker.id=1
listeners=PLAINTEXT://:9093
log.dir=/tmp/kafka-logs-1
config/server-2.properties:
broker.id=2
listeners=PLAINTEXT://:9094
log.dir=/tmp/kafka-logs-2
我们已经有Zookeeper并启动了单个节点,因此我们只需要启动两个新节点:
> bin/kafka-server-start.sh config/server-1.properties &
> bin/kafka-server-start.sh config/server-2.properties &
现在创建一个复制因子为3的新主题以及3个分区 ,它将解决此问题:
> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic my-replicated-topic
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.