[英]How to trigger a pipeline in Azure Data Factory v2 or a Azure Databricks Notebook by a new file in Azure Data Lake Store gen1
I am using a Azure Data Lake Store gen1 for storing JSON files.我正在使用Azure Data Lake Store gen1来存储 JSON 文件。 Based on these files i have Notebooks in Azure Databricks for processing them.
基于这些文件,我在Azure Databricks 中有 Notebooks 来处理它们。 Now i want to trigger such a Azure Databricks Notebook when a new file is creating in Azure Data Lake Store gen1 .
现在我想在Azure Data Lake Store gen1 中创建新文件时触发这样的Azure Databricks Notebook。 I couldnt find any Trigger which could do this.
我找不到任何可以做到这一点的触发器。 do you know any way?
你知道什么方法吗?
Currently, this is not yet implemented/Supported by Microsoft .目前,这尚未由Microsoft实施/支持。 But it is on their Roadmap(I believe).
但这在他们的路线图上(我相信)。
You can do this in 2 ways,你可以通过两种方式做到这一点,
Azure Functions(through Event Grid) Azure 函数(通过事件网格)
Logic Apps逻辑应用
Currently, Microsoft is building on #1.目前,微软正在构建#1。
You can track the issue here .您可以在此处跟踪问题。
As per this按照这个
This feature is not a high priority for us right now, but I will note that the announcement for Azure Event Grid listed Data Lake as one of the integrations they are building.
这个功能目前对我们来说不是一个高优先级,但我会注意到 Azure 事件网格的公告将数据湖列为他们正在构建的集成之一。 Once you can subscribe to Data Lake updates through Event Grid, running an Azure Function would be trivial (see here for some info).
一旦您可以通过事件网格订阅数据湖更新,运行 Azure 函数将是微不足道的(请参阅此处了解一些信息)。
You can vote your voice to support the event grid (provider) in DataLake.您可以投票支持 DataLake 中的事件网格(提供者)。
This is also not yet implemented, but you can Upvote your voice here to support this feature这也尚未实现,但您可以在此处为您的声音点赞以支持此功能
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.