简体   繁体   中英

how to write kafka to kafka sink connector

i was trying for kafka mirror. And i found this open source [Comcast]: https://github.com/Comcast/MirrorTool-for-Kafka-Connect and it is successfully working when retrieving data form source kafka topic and writing it to destination kafka topic. now i need to sink from destination kafka topic and writing it to source kafka. How can i do that please suggest references.

i wrote kafkaSinkTask file. and inside put method i am able to get topics from destination kafka. so i don't know how to write this topics to source kfafa

    @Override
    public void put(Collection<SinkRecord> records) {
        if (records.isEmpty()) {
              return;
            }
            final SinkRecord first = records.iterator().next();
            final int recordsCount = records.size();
            logger.info(
                "Received {} records. First record kafka coordinates:({}-{}-{}).
                + "database...",
                recordsCount, first.topic(), first.kafkaPartition(), first.kafkaOffset()
            );  
    }

now i need to sink from destination kafka topic and writing it to source kafka

Can you not just flip the source and destination servers in your config file? Connect should generally consume from the remote cluster and produce to a local one (given two data centers or geographically separated networks), and store the consumed offsets at the destination in the Connect offsets topic, thus it being a source connector.

You could also try MirrorMaker 2.0 instead - https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM