如何使用 Nest Js 将消息发送到 Kafka 中的自定义分区

Question

I try to send message into right partition here is my NESTJS Producer Controller

 constructor(
    private readonly appService: AppService,
    @Inject('any_name_i_want') private readonly client: ClientKafka,
  ) {}

  async onModuleInit() {
    ['test2'].forEach((key) => this.client.subscribeToResponseOf(`${key}`));
    await this.client.connect();
  }

@Get('kafka-test-with-response')
  async testKafkaWithResponse() {
    const res = await this.client.send('test2', {
      foo: 'bar',
      data: new Date().toString(),
      partition: 1,
    });

    return res;
  }

I try to read in NestJs Document but it's provide link into Kafka website and I've found this in Kafka document

const producer = kafka.producer()

await producer.connect()
await producer.send({
    topic: 'topic-name',
    messages: [
        { key: 'key1', value: 'hello world', partition: 0 },
        { key: 'key2', value: 'hey hey!', partition: 1 }
    ],
})

After I've check in UI Kafka enter image description here

It seems like Producer still send random partition

I try to deep down check in typescript send() function but it use any so I'm really not sure how to specify partition in send function

Answer 1

NestJS does use KafkaJS, so that code section you found is correct, assuming you could get a raw kafkajs client instance.

Alternatively, Kafka uses the record key to partition each event, and in the NestJS ProducerConfig, you can specify a partitioner instance. https://github.com/nestjs/nest/blob/master/packages/microservices/external/kafka.interface.ts#L104