简体   繁体   English

对于翻滚窗口,emitEarlywhenFull()的抑制如何工作?

[英]How does the suppress with emitEarlywhenFull() for tumbling windows works?

I am using suppress on tumbling windows to get aggregated results. 我在翻滚窗口上使用抑制来获得聚合结果。 I am exploring both untilTimeLimit and untilWindowCloses for suppress. 我正在探索untilTimeLimit和untilWindowCloses进行压制。 I dont want my streams to shutdown when buffer fulls. 当缓冲区满时,我不希望我的流关闭。 I have seen this feature emitEarlyWhenFull() , but it cant be applicable on top of untilWindowCloses. 我已经看到这个函数emitEarlyWhenFull(),但它不能适用于untilWindowCloses。 Hence, i am picking untilTimeLImit with emitEarlyWhenFull() , please refer below code : 因此,我使用emitEarlyWhenFull()选择untilTimeLImit,请参考下面的代码:

groupedStreams.windowedBy(TimeWindows.of(Duration.ofMinutes(5)))    
.aggregate(() -> initialBlob, blobAggregator,someserde)
.suppress(Suppressed.untilTimeLimit(Duration.ofMinutes(5), new StrictBufferConfigImpl().emitEarlyWhenFull()))
.toStream()

In my case, I am using tumbling windows for 5 mins. 在我的情况下,我使用翻滚窗户5分钟。 So, for every 5 mins, a window will be open for every record key. 因此,每隔5分钟,将为每个记录键打开一个窗口。 According to documentation, oldest records will be sent when the buffer gets filled. 根据文档,缓冲区填满时将发送最旧的记录。 what happens to the new records with same key comes after old records sent down in the same tumbling window? 在同一个翻滚窗口中发送旧记录之后,具有相同密钥的新记录会发生什么?

For Example : messages flow : (A,1) (A,2) (A,3) -> agg result : (A,6) . 例如:消息流:(A,1)(A,2)(A,3) - > agg结果:(A,6)。 suppose here , the buffer is full, (A,6) will be sent downstream. 假设这里缓冲区已满,(A,6)将向下游发送。 Lets suppose (A,4) comes now in the same tumbling window, what will come next ? 让我们假设(A,4)现在出现在同一个翻滚窗口中,接下来会发生什么? will it be : (A,10) or it will start fresh with (A,4) again? 它会是:(A,10)还是会再次以(A,4)重新开始?

If suppress() emits, the state will be preserved. 如果suppress()发出,则将保留状态。 Thus, for your example, the aggregation will continue and eventually (A,10) will be emitted. 因此,对于您的示例,聚合将继续并最终(A,10)将被发出。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 窗口翻转在ksql中如何工作? 作为查询返回相同的结果,无论是否在ksql中使用窗口滚动 - How does window tumbling works in ksql? As query returning same result with or without using window tumbling in ksql 使用翻滚窗口时如何选择时间语义的类型? - How to select the type of time semantics when working with tumbling windows? 是否可以使用输入流中的时间戳在 Flink 中创建 Tumbling Windows。 如果是,那么如何? - Is it possible to create Tumbling Windows in Flink using a timestamp coming in Input Stream. If yes then How? 如何对 Tumbling Window 应用偏移量,以延迟 Windows 的启动<timewindow>在卡夫卡流中</timewindow> - How to apply an offset to Tumbling Window, in order to delay the starting of Windows<TimeWindow> in Kafka Streams Flink 中非 keyed 流数据如何使用 Tumbling 窗口功能? - How to use the Tumbling window function for the non keyed streaming data in Flink? Kafka Stream如何使用KTable#Suppress发送最终聚合? - How does Kafka Stream send final aggregation with KTable#Suppress? 使用窗口翻转时如何从 KSQL 流中获取最后一个值 - How to get last value from a KSQL stream when using window tumbling 物联网 - KSQL 翻滚问题 - IoT - KSQL Tumbling issues .poll方法如何工作? - How does .poll method works? 重试逻辑在 kafka 生产者中如何工作? - How does the retry logic works in kafka producers?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM