[英]Azure Data Factory -> Lookup , ForEach and copy activity
I want to take a file from the azure synapse and load it in ADLs using ADf.我想从 azure 突触中获取一个文件,并使用 ADf 将其加载到 ADL 中。 I want to read the data of the Last 13 months and make a different file for each month.
我想读取过去 13 个月的数据并为每个月制作一个不同的文件。 I made a CSV file where I wrote the start date and end date of each month and make a lookup activity over this file.
我制作了一个 CSV 文件,我在其中写下了每个月的开始日期和结束日期,并对这个文件进行了查找活动。 Then using foreach activity, I load the file from the copy activity.
然后使用 foreach 活动,我从复制活动中加载文件。 Now I want to write a query for each month's data.
现在我想为每个月的数据编写一个查询。
select * from edw.factbaldly where INSERT_DATE > @activity('Lookup1').output.value.startdate and INSERT_DATE < @activity('Lookup1').output.value.EndDate select * from edw.factbaldly where INSERT_DATE > @activity('Lookup1').output.value.startdate and INSERT_DATE < @activity('Lookup1').output.value.EndDate
select * from edw.factbaldly where INSERT_DATE > @item().startdate and INSERT_DATE < @item().EndDate' select * 来自 edw.factbaldly 其中 INSERT_DATE > @item().startdate 和 INSERT_DATE < @item().EndDate'
I use these to queries but not able to read the data of lookup activity and fetch the data.我使用这些查询但无法读取查找活动的数据并获取数据。 Please help me with the query.
请帮我查询。 Thanks in advance.
提前致谢。
I assume your Lookup1 CSV column headings are startdate
and enddate
我假设您的 Lookup1 CSV 列标题是
startdate
和enddate
In your ForEach > Settings > Items you will have @activity('Lookup1').output.value
在您的 ForEach > Settings > Items 您将拥有
@activity('Lookup1').output.value
Inside the ForEach block, your Copy activity Source query will look like:在 ForEach 块内,您的 Copy 活动源查询将如下所示:
select * from edw.factbaldly where INSERT_DATE > '@{item().startdate}' and INSERT_DATE < '@{item().enddate}'
ADF will substitute @{thing}
with a string so you'll get the dates as quoted strings in the query ADF 会将
@{thing}
替换为字符串,以便您在查询中将日期作为带引号的字符串
Maybe also you want one of the signs as >= or <=?也许您还想要 >= 或 <= 的符号之一?
In fact you probably don't need to maintain the CSV because you can use a variable and ADF functions utcnow(), addToTime() and startOfMonth() to find the dates事实上,您可能不需要维护 CSV,因为您可以使用变量和 ADF 函数 utcnow()、addToTime() 和 startOfMonth() 来查找日期
In the lookup activity you will fetch the @item().startdate and @item().EndDate.在查找活动中,您将获取@item().startdate 和@item().EndDate。 Or I guess you have already set those in the lookup before ForEach.
或者我猜你已经在 ForEach 之前的查找中设置了这些。 But to use this details when you produce new files, you must use the query from the question in source part of the Copy Activity.
但是要在生成新文件时使用此详细信息,您必须使用复制活动源部分中问题的查询。
If you can't use the query directly on the file, you can import the whole file to DB table and then use your query in the copy activity source.如果您不能直接在文件上使用查询,您可以将整个文件导入数据库表,然后在复制活动源中使用您的查询。
You can use an expression like this您可以使用这样的表达式
@concat('select * from edw.factbaldly where INSERT_DATE> >',item().startdate,'and INSERT_DATE <',item().EndDate)
If I where you, i could have added a set variable activity and tested the above expression.如果我在你那里,我可以添加一个设置变量活动并测试上面的表达式。 The set variable should give us a syntactically correct TSQL statement.
set 变量应该给我们一个语法正确的 TSQL 语句。 HTH
高温高压
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.