简体   繁体   English

如何将 kafka 写入 kafka 接收器连接器

[英]how to write kafka to kafka sink connector

i was trying for kafka mirror.我正在尝试 kafka 镜像。 And i found this open source [Comcast]: https://github.com/Comcast/MirrorTool-for-Kafka-Connect and it is successfully working when retrieving data form source kafka topic and writing it to destination kafka topic.我发现了这个开源 [Comcast]: https : //github.com/Comcast/MirrorTool-for-Kafka-Connect ,它在从源 kafka 主题检索数据并将其写入目标 kafka 主题时成功工作。 now i need to sink from destination kafka topic and writing it to source kafka.现在我需要从目标 kafka 主题下沉并将其写入源 kafka。 How can i do that please suggest references.我该怎么做,请建议参考。

i wrote kafkaSinkTask file.我写了kafkaSinkTask文件。 and inside put method i am able to get topics from destination kafka.在 put 方法中,我可以从目标 kafka 获取主题。 so i don't know how to write this topics to source kfafa所以我不知道如何写这个主题来获取 kfafa

    @Override
    public void put(Collection<SinkRecord> records) {
        if (records.isEmpty()) {
              return;
            }
            final SinkRecord first = records.iterator().next();
            final int recordsCount = records.size();
            logger.info(
                "Received {} records. First record kafka coordinates:({}-{}-{}).
                + "database...",
                recordsCount, first.topic(), first.kafkaPartition(), first.kafkaOffset()
            );  
    }

now i need to sink from destination kafka topic and writing it to source kafka现在我需要从目标 kafka 主题下沉并将其写入源 kafka

Can you not just flip the source and destination servers in your config file?你不能只是在你的配置文件中翻转源服务器和目标服务器吗? Connect should generally consume from the remote cluster and produce to a local one (given two data centers or geographically separated networks), and store the consumed offsets at the destination in the Connect offsets topic, thus it being a source connector. Connect 通常应该从远程集群消费并生产到本地集群(给定两个数据中心或地理上分离的网络),并将消耗的偏移量存储在 Connect offsets 主题中的目的地,因此它是一个源连接器。

You could also try MirrorMaker 2.0 instead - https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0您也可以尝试使用 MirrorMaker 2.0 - https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM