[英]Kafka Producer Guidelines java developers
I am a beginner of kafka implementation.我是kafka实施的初学者。 Can you give me some guidelines for writing a kafka producer to implement below requirements?
你能给我一些编写 kafka 生产者以实现以下要求的指导方针吗? I am setting up kafka topics with avro schema.
我正在使用 avro 模式设置 kafka 主题。
3.if any consumer joined to subscribe my topics very late, may be after 8 months after my application deployed in prod, he should be able to consume all data whatever before published.Is this possible to maintain more months in topics? 3.如果有任何消费者很晚才加入订阅我的主题,可能是在我的应用程序部署到生产环境后 8 个月之后,他应该能够使用发布之前的所有数据。这是否可以在主题中维护更多月份? If so how do we do that to inline with first two requirements.
如果是这样,我们如何做到这一点以符合前两个要求。
Please Suggest some ideas really appreciated.请提出一些非常感谢的想法。 Thanks
谢谢
Currently i am writing a poc to brush up skills a sample producer which publish messages to kafka topic for any update on data.目前,我正在编写一个 poc 来提高样本制作人的技能,该制作人将消息发布到 kafka 主题以获取任何数据更新。 I am expecting ideal case for pushing messages to kafka topics
我期待将消息推送到 kafka 主题的理想情况
Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, io.confluent.kafka.serializers.KafkaAvroSerializer.class);
props.put("schema.registry.url", "http://localhost:8081");
KafkaProducer producer = new KafkaProducer(props);
In my opinion it is better to publish always entire event to Kafka.在我看来,最好始终将整个事件发布到 Kafka。 Considering fact that you are using Avro you don't need to care about handling multiple null fields in your scheme or storing different schemas for same topic.
考虑到您正在使用 Avro,您无需关心处理方案中的多个 null 字段或为同一主题存储不同的方案。 Or if consumer will decide to consume from specific offset it will always has access to latest version of the event.
或者,如果消费者决定从特定偏移量消费,它将始终可以访问最新版本的事件。
You can keep record in Kafka unlimited amount of time if you set retention.ms
-1
如果你设置
retention.ms
-1
,你可以在 Kafka 中保持记录无限的时间
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.