[英]Azure Stream Analytics Job delayed output of not matching records by 1 minutes between two event hubs
Could anyone help me, why the not matching records are delayed by exactly 1 minutes but the matching records are writing into blog storage container immediately.任何人都可以帮助我,为什么不匹配的记录会延迟 1 分钟,但匹配的记录会立即写入博客存储容器。
Is there any way to avoid the delay even though eventA its not matching with other eventB (being my downstream system will take care in my use-case)即使 eventA 与其他 eventB 不匹配,是否有任何方法可以避免延迟(作为我的下游系统将在我的用例中处理)
select b.eventenqueuedutctime as btime,a.Id,a.SysTime,a.UTCTime
,b.Id as BId,b.SysTime as BSysTime
into outputStorage -- to blob storage (container)
from eventA a TIMESTAMP BY eventenqueuedutctime
left outer join eventB b TIMESTAMP BY eventenqueuedutctime
on a.id = b.id
and datediff(minute,b,a) between 0 and 180 -- join with last 3 hours of eventB data
Below is the output but look at the last row (Id:99) currentTime:T19: 42:13.1690000 Z which delayed by 1 minute compared top 4 rows (currentTime:T19: 41:13.1690000 Z)下面是输出,但看看最后一行 (Id:99) currentTime:T19: 42:13.1690000 Z 与前 4 行相比延迟了 1 分钟 (currentTime:T19: 41:13.1690000 Z)
FYI, Sending all the eventA Id (2,4,1,101,99) at once via EventDataBatch via Json serialization仅供参考,通过 Json 序列化通过 EventDataBatch 一次发送所有 eventA Id (2,4,1,101,99)
{"btime":"2020-11-03T17:00:50.6360000Z","Id":2,"SysTime":"2020-11-03T11:41:12.860466-08:00","UTCTime":"2020-11-03T19:41:12.8604646Z","BId":2,"BSysTime":"2020-11-03T09:00:49.6751336-08:00","fullname":"cc","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":4,"SysTime":"2020-11-03T11:41:12.8605138-08:00","UTCTime":"2020-11-03T19:41:12.8605135Z","BId":4,"BSysTime":"2020-11-03T09:00:49.6751371-08:00","fullname":null,"currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":1,"SysTime":"2020-11-03T11:41:12.8605561-08:00","UTCTime":"2020-11-03T19:41:12.8605559Z","BId":1,"BSysTime":"2020-11-03T09:00:49.6749841-08:00","fullname":"test","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T19:39:04.0100000Z","Id":101,"SysTime":"2020-11-03T11:41:12.860598-08:00","UTCTime":"2020-11-03T19:41:12.8605978Z","BId":101,"BSysTime":"2020-11-03T11:39:03.7462454-08:00","fullname":"test-101","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":null,"Id":99,"SysTime":"2020-11-03T11:41:12.860322-08:00","UTCTime":"2020-11-03T19:41:12.8602803Z","BId":null,"BSysTime":null,"fullname":null,"currentTime":"2020-11-03T19:42:13.1690000Z"}
This is because you use JOIN with DATEDIFF.这是因为您将 JOIN 与 DATEDIFF 一起使用。
The use of temporal joins, such as JOIN with DATEDIFF:
临时连接的使用,例如 JOIN with DATEDIFF:
Matches generate as soon as both sides of the matched events arrive.
一旦匹配事件的双方到达,匹配就会生成。
Data that lacks a match, like LEFT OUTER JOIN, is generated at the end of the DATEDIFF window, for each event on the left side.
对于左侧的每个事件,在 DATEDIFF 窗口的末尾生成缺少匹配项的数据,例如 LEFT OUTER JOIN。
More details, you can refer to https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-troubleshoot-output#the-first-output-is-delayed .更多详情,可以参考https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-troubleshoot-output#the-first-output-is-delayed 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.