[英]Do Kafka Streams have a timeout on how long processing can take?
I'm using Kafka Streams that call external systems during map/foreach
. 我正在使用在map/foreach
期间调用外部系统的Kafka Streams。 Is there any timeout on how long map
or foreach
can take? map
或foreach
可以花费多长时间?
Are there any caveats to blocking for a long time (say hours)? 在长时间(例如几个小时)内是否有要阻止的警告?
There is no timeout applied to map/foreach
. 没有超时适用于map/foreach
。
However, as Kafka Streams uses KafkaConsumer
and KafkaProducer
internally, all their timeouts apply (eg. max.poll.interval.ms
). 但是,由于Kafka Streams在内部使用KafkaConsumer
和KafkaProducer
,因此它们的所有超时都适用(例如max.poll.interval.ms
)。 You can of course configure them accordingly, but it's not recommended to do long blocking calls to external systems. 您当然可以对其进行相应的配置,但是不建议您长时间阻止对外部系统的调用。
It would have many advantages, to load the data from you external service into a topic using Kafka Connect, and read this data as a KTable
in your application a do a stream-table join instead of a map. 使用Kafka Connect将外部服务中的数据加载到主题中,并在应用程序中以KTable
读取此数据, KTable
执行流表KTable
而不是映射,这将具有很多优点。 This approach decouples your Streams application from the external system and thus makes the overall architecture more robust. 这种方法使您的Streams应用程序与外部系统脱钩,从而使整体体系结构更强大。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.