Kafka Streams: How to transform message before publishing to different topics

Question

Messages on Topic A are read using Kafka Streams. Based on some validation, the message is published to either Topic B or Topic C. This was easily achieved using Kafka Streams' branch method

KStreamBuilder kStreamBuilder = new KStreamBuilder();
KStream<String, String> kStream = kStreamBuilder.stream(Serdes.String(), Serdes.String(), topicA.split(","));

KStream<String, String>[] splitStreams = kStream.branch(
    (key, value) -> process(value), //Process method is expensive
    (key, value) -> true
);

splitStreams[0].to(Serdes.String(), Serdes.String(), topicB);
splitStreams[1].to(Serdes.String(), Serdes.String(), topicC);

Problem: Based on whether the message passes or fails validation, it is to be edited/appended before publishing it to either Topic B or Topic C. How do I achieve that?

Answer 1

for transforming messages you could use the following KStream methods:

• map (for updating message based by key & value)
• mapValues (for updating message based by value)
• selectKey (for updating key based by key & value)

example:

splitStreams[0].mapValues(value -> transformValueForTopicB(value))
        .to(Serdes.String(), Serdes.String(), topicB);
splitStreams[1].map((key, value) -> transformValueForTopicC(key, value))
        .to(Serdes.String(), Serdes.String(), topicC);

also as I see from your process method name inside branch method, potentially you try to process value, but you only need to provide Predicate (that returns true or false ) without any processing.