Azure 流分析作业将两个事件中心之间不匹配记录的输出延迟了 1 分钟

Question

任何人都可以帮助我，为什么不匹配的记录会延迟 1 分钟，但匹配的记录会立即写入博客存储容器。

即使 eventA 与其他 eventB 不匹配，是否有任何方法可以避免延迟（作为我的下游系统将在我的用例中处理）

select b.eventenqueuedutctime as btime,a.Id,a.SysTime,a.UTCTime
,b.Id as BId,b.SysTime as BSysTime 
into outputStorage -- to blob storage (container)
from eventA a TIMESTAMP BY eventenqueuedutctime
left outer join eventB b TIMESTAMP BY eventenqueuedutctime
on a.id = b.id
and datediff(minute,b,a) between 0 and 180 -- join with last 3 hours of eventB data

下面是输出，但看看最后一行 (Id:99) currentTime:T19: 42:13.1690000 Z 与前 4 行相比延迟了 1 分钟 (currentTime:T19: 41:13.1690000 Z)

仅供参考，通过 Json 序列化通过 EventDataBatch 一次发送所有 eventA Id (2,4,1,101,99)

{"btime":"2020-11-03T17:00:50.6360000Z","Id":2,"SysTime":"2020-11-03T11:41:12.860466-08:00","UTCTime":"2020-11-03T19:41:12.8604646Z","BId":2,"BSysTime":"2020-11-03T09:00:49.6751336-08:00","fullname":"cc","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":4,"SysTime":"2020-11-03T11:41:12.8605138-08:00","UTCTime":"2020-11-03T19:41:12.8605135Z","BId":4,"BSysTime":"2020-11-03T09:00:49.6751371-08:00","fullname":null,"currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":1,"SysTime":"2020-11-03T11:41:12.8605561-08:00","UTCTime":"2020-11-03T19:41:12.8605559Z","BId":1,"BSysTime":"2020-11-03T09:00:49.6749841-08:00","fullname":"test","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T19:39:04.0100000Z","Id":101,"SysTime":"2020-11-03T11:41:12.860598-08:00","UTCTime":"2020-11-03T19:41:12.8605978Z","BId":101,"BSysTime":"2020-11-03T11:39:03.7462454-08:00","fullname":"test-101","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":null,"Id":99,"SysTime":"2020-11-03T11:41:12.860322-08:00","UTCTime":"2020-11-03T19:41:12.8602803Z","BId":null,"BSysTime":null,"fullname":null,"currentTime":"2020-11-03T19:42:13.1690000Z"}

Answer 1

这是因为您将 JOIN 与 DATEDIFF 一起使用。

临时连接的使用，例如 JOIN with DATEDIFF：

一旦匹配事件的双方到达，匹配就会生成。

对于左侧的每个事件，在 DATEDIFF 窗口的末尾生成缺少匹配项的数据，例如 LEFT OUTER JOIN。

更多详情，可以参考https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-troubleshoot-output#the-first-output-is-delayed 。