如果我有多个生产者向 Kafka 生产相同的数据，为每个生产者配置一个主题或为所有生产者配置一个主题是否有效

Question

I have more number of producers let's say around 200. and each producer have 4 different kinds of data. so is it efficient to produce data from all producers to same topic or is it efficient to configure different topic for each producer?

I want to collect each producer data separately at the consumer end.

What are the available ways to handle data at the consumer end?

Answer 1

There's no difference. The broker just stores bytes; it has no knowledge of "type".

The main problem with putting everything into one topic is that your consumers will need to know how to deserialize and process those bytes, consistently. Ie setting value.deserializer might not return a concrete class, but rather some generic type like JSON string or Avro GenericRecord, which then needs extra parsing and large switch-case logic in your consumer