如何將數據發送到特定的 Kafka 主題分區？

Question

我有這個簡單的生產者，主題有 2 個分區。 我想將前十條消息發送到分區 0，其余十條消息發送到另一個分區。 有什么辦法可以嗎？

 package com.company; import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.Producer; import org.apache.kafka.clients.producer.ProducerRecord; import java.util.Properties; public class Main { public static void main(String[] args) { Properties properties = new Properties(); properties.put("bootstrap.servers", "localhost:9092"); properties.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer"); properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); Producer<Integer, String> producer = new KafkaProducer<>(properties); for (int i = 0; i <= 20; i++) { ProducerRecord<Integer, String> producerRecord = new ProducerRecord<>("DemoTopic", i, "Test Message" + i); producer.send(producerRecord); } producer.close(); } }

Answer 1

您將不得不編寫一個自定義分區器！

import java.util.Map;
import org.apache.kafka.clients.producer.Partitioner;
import org.apache.kafka.common.Cluster;
 
public class KafkaIntegerPartitioner implements Partitioner {

 
  @Override
  public void configure(Map<String, ?> configs) {
 
  }
 
  @Override
  public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
     int partition = 0;
    Integer keyVal = (Integer)key;
    //<logic for partition>
    int divident = keyVal/10;
    if(divident%2==0){
        return 0;
    }
    return 1;
  }
 
  @Override
  public void close() {
 
  }
} 

然后在生產者

properties.put("partitioner.class","KafkaIntegerPartitioner");

Answer 2

您只需使用實際接受分區的不同構造函數

int count = 0; ProducerRecord<Integer, String> producerRecord; for (int i = 0; i <= 20; i++) { int partition = count < 10? 0: 1; producerRecord = new ProducerRecord("topic", partition, i, "Message " + i);... count++; }