[英]How to perform Event based data ingestion using Azure Data Lake Storage Gen2 and Azure Data factory V2?
Recently we came across a scenario where our source and sink location are of ADLS Gen2 type.最近我们遇到了一个场景,其中我们的源和接收器位置是 ADLS Gen2 类型。 Now we got one interesting use case wherein we have to push data from source to sink with the help of ADF V2.
现在我们有了一个有趣的用例,我们必须在 ADF V2 的帮助下将数据从源推送到接收器。 Having said that, its not just normal copy activity we are expecting but we need to perform this activity on an event basis.
话虽如此,这不仅仅是我们期望的正常复制活动,而是我们需要在事件基础上执行此活动。
While going through the ADLS Gen2 documents found that ADLS Gen2 yet to support "Azure Event Grids" and that's the reason though we are able to configure ADF's event-based triggers they did not work.在浏览 ADLS Gen2 文档时发现 ADLS Gen2 尚未支持“Azure 事件网格”,这就是我们能够配置 ADF 的基于事件的触发器但它们不起作用的原因。
Can anyone suggest me to tackle this situation, since Azure Event Gird is not supported at this instance of time we don't believe we can achieve this with Azure Event Hubs and their integration with ADF?任何人都可以建议我解决这种情况,因为此时不支持 Azure Event Gird,我们不相信我们可以通过 Azure 事件中心及其与 ADF 的集成来实现这一点?
Thanks.谢谢。
From my repro, currently event based trigger are supported only on v2 storage accounts.根据我的重现,当前基于事件的触发器仅在 v2 存储帐户上受支持。
Data Factory is now integrated with Azure Event Grid, which lets you trigger pipelines on an event.数据工厂现已与 Azure 事件网格集成,可让您在事件上触发管道。
Note: This integration supports only version 2 Storage accounts (General purpose).注意:此集成仅支持版本 2 存储帐户(通用)。
Azure Event Grid doesn't receive events from Azure Data Lake Gen2 accounts because those accounts don't yet generate them. Azure 事件网格不会从 Azure Data Lake Gen2 帐户接收事件,因为这些帐户尚未生成它们。
For more details, refer “ Known issues with Azure Data Lake Storage Gen2 ”.有关更多详细信息,请参阅“ Azure Data Lake Storage Gen2 的已知问题”。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.