Kafka multiple Topics into the same avro file

Question

i am new of the KAFKA protocol's world and i would like to ask you some inportant information related to my project.

I am using AVRO file for producing and consuming messages, i want to know if i can use the same avro file for multiple Topics maybe for example by using a different "name" attribute into the producer and by using a specific "name" attribute in the consumer.

Thanks a lot. Stefano

Answer 1

You can use one file to send data to multiple topics, yes, although I'm not sure why one would do that

I would be cautious about merging multiple topics into one Avro file because the schema must match in every topic for that file

It would be suggested that you use the Confluent Schema Registry, for example, rather than sending individual Avro events because if you are not using some registry, then you're likely sending the Avro schema as part of every message, which will slow down the possible throughput of your topic. And then, the name of the Avro schema record in the register will correspond to the topic name