简体   繁体   中英

kafka streams - NullPointerException when joining KStream with KTable leftjoin()

I use leftJoin to transform a stream message (transaction) using values from tables. I concat multiple joins. After every join I select a new key. I am getting NullPointerException in my last join. Filtering null keys does not help.

Exception in thread "account-transactions-e0a66751-f65e-4ba0-947c-024b5d32f7c2-StreamThread-1" java.lang.NullPointerException at org.namematching.kafkastreams.transactiontransformation.TransactionTransformationStream.lambda$launchStream$14(TransactionTransformationStream.java:124)
at org.apache.kafka.streams.kstream.internals.KStreamKTableJoinProcessor.process(KStreamKTableJoinProcessor.java:58)
at org.apache.kafka.streams.processor.internals.ProcessorNode$1.run(ProcessorNode.java:46)
at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:208)
at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:124)
at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:85)
at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:80)
at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:216)
at org.apache.kafka.streams.processor.internals.AssignedTasks.process(AssignedTasks.java:403)
at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:317)
at org.apache.kafka.streams.processor.internals.StreamThread.processAndMaybeCommit(StreamThread.java:942)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:822)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:774)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:744)

Here is my join logic:

KStream<String,DoiTAccountTransaction> transactionsWithAccounts =
            transactionStream.leftJoin(accountTable,(v1,v2) -> { v1.setBeneficiaryBankAccount(v2.getOriginId()); return v1; }, Joined.with(Serdes.String(), transactionSerde, accountSerde))
            .selectKey((k,v) -> v.getTransactionCode());

KStream<String,DoiTAccountTransaction> transactionsWithTransactionCodes =
            transactionsWithAccounts.leftJoin(transactionCodesTable, (v1,v2) -> {v1.setTransactionCode(v2.getUid()); return v1; }, Joined.with(Serdes.String(), transactionSerde, transactionCodesSerde))
            .selectKey((k,v) -> v.getCurrency());

KStream<String,DoiTAccountTransaction> transactionsWithCurrencies =
            transactionsWithTransactionCodes.leftJoin(currenciesTable, (v1,v2) -> {v1.setCurrency(v2.getUid()); return v1;}, Joined.with(Serdes.String(), transactionSerde, currenciesSerde))
            .selectKey((k,v) -> v.getOriginalBeneficiary1());

KStream<String, DoiTAccountTransaction> transactionsWithPersons =
            transactionsWithCurrencies.leftJoin(personsTable,(v1,v2) -> {v1.setOriginalBeneficiary1(v2.getUid()); return v1;}, Joined.with(Serdes.String(), transactionSerde, personsSerde));


transactionsWithPersons.to("account-transactions-processed", Produced.with(Serdes.String(),transactionSerde));

Last join fails. Filtering null keys does not help.

Is it possible that a row in the personsTable for that key was not present (at that time) and the NPE was coming from the v2.getUid() expression?

It would be safer to do a not null check on all the right hand side objects in the joins before accessing their fields (all the v2's).

https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Join+Semantics#KafkaStreamsJoinSemantics-KStream-KTableJoin

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM