[英]Changing a Date automatically every month in copy activity in Azure Data Factory to copy data from SAP HANA to Azure SQLDB
I am working in azure data factory (ADF) to copy data from SAP HANA to Azure SQLDB .我在azure 数据工厂 (ADF) 工作以将数据从SAP HANA复制到Azure SQLDB 。 I am using "copy activity" in ADF to perform this operation using SQL query to query the table in HANA and sink it into SQLDB.我在 ADF 中使用“复制活动”来执行此操作,使用 SQL 查询来查询 HANA 中的表并将其沉入 SQLDB。 The query contains reporting_date
which I need to manually update every month to fetch the data.查询包含reporting_date
,我需要每月手动更新以获取数据。
I want to remove this redundant procedure of changing dates manually and then run it every month because i have 350+ pipelines to change dates every month and this task becomes very hectic and time consuming.我想删除这个手动更改日期的冗余程序,然后每个月运行它,因为我每个月都有 350 多个管道来更改日期,而这项任务变得非常忙碌和耗时。 Is there any way to change the date at one place, it works as an input to the source query and query gets automatically updated with the date?有什么方法可以在一个地方更改日期,它可以作为源查询的输入,并且查询会随着日期自动更新? Please find attached screenshots for my requirement.请根据我的要求找到附加的屏幕截图。
Query to be updated:要更新的查询:
SELECT "SAPINSTANCE", "MCO", "COUNTRY", "BUKRS", "EntityName", "XBLNR", "HKONT", "TXT50", "GSBER", "BELNR", "BLART", "LTEXT", "CC_BLDAT", "CC_BUDAT", "WAERS", "MONAT", "SGTXT", "GJAHR", "BKTXT", "UMSKZ", "S_LTEXT", "AUFNR", "PROJK", "PRCTR", "EBELN", "KOART", "AUGBL", "CC_AUGDT", "REBZG", "LIFNR", "NAME1", "CC_SHKZG", "KOSTL", "BSCHL", "BUZEI", "EBELP", "KTOKS", "ZUONR", "XINTB", "XLOEB", "XSPEB", "XOPVW", "XKRES", "TYPE_WISE_CLASS", "DESCRIPTION", "MCLASS", "CC_NUM_OF_DAYS_AGEING", "CC_AGEING_BUCKET", "CC_GROUP_CURRENCY", "CC_REPORTING_DATE_VAR", "MARKET_CLUSTER", "HWAER", "CURR_UNIT_TO_EURO_1", sum("CC_WRBTR") AS "CC_WRBTR", sum("CC_GROUP_CURRENCY_VALUE") AS "CC_GROUP_CURRENCY_VALUE", sum("CC_LOC_AMT_IN_MIN") AS "CC_LOC_AMT_IN_MIN", sum("CC_AMT_IN_EUR_MIN") AS "CC_AMT_IN_EUR_MIN", sum("CC_DMBTR_E") AS "CC_DMBTR_E"
FROM "_SYS_BIC"."table"('PLACEHOLDER' = ('$$IP_CLUSTER$$', 'Africa'), 'PLACEHOLDER' = ('$$IP_COUNTRY$$', '*'), 'PLACEHOLDER' = ('$$IP_FISCAL_YEAR$$', '*'), 'PLACEHOLDER' = '$$IP_REPORTING_DATE$$', '20200831'), 'PLACEHOLDER' = ('$$IP_AGEING_BUCKET$$', '''*'''), 'PLACEHOLDER' = ('$$IP_BUKRS$$', '*'))
GROUP BY "SAPINSTANCE", "MCO", "COUNTRY", "BUKRS", "EntityName", "XBLNR", "HKONT", "TXT50", "GSBER", "BELNR", "BLART", "LTEXT", "CC_BLDAT", "CC_BUDAT", "WAERS", "MONAT", "SGTXT", "GJAHR", "BKTXT", "UMSKZ", "S_LTEXT", "AUFNR", "PROJK", "PRCTR", "EBELN", "KOART", "AUGBL", "CC_AUGDT", "REBZG", "LIFNR", "NAME1", "CC_SHKZG", "KOSTL", "BSCHL", "BUZEI", "EBELP", "KTOKS", "ZUONR", "XINTB", "XLOEB", "XSPEB", "XOPVW", "XKRES", "TYPE_WISE_CLASS", "DESCRIPTION", "MCLASS", "CC_NUM_OF_DAYS_AGEING", "CC_AGEING_BUCKET", "CC_GROUP_CURRENCY", "CC_REPORTING_DATE_VAR", "MARKET_CLUSTER", "HWAER", "CURR_UNIT_TO_EURO_1"
The above query acts as an input in the copy activity and needs to be updated with date every month manually.上述查询作为复制活动的输入,需要每个月手动更新日期。 Please help!请帮忙!
I presume that you want to use "current_date" or similar dynamic value when triggering Pipeline execution using a scheduled event (for instance weekly, or monthly).我假设您想在使用预定事件(例如每周或每月)触发流水线执行时使用“current_date”或类似的动态值。
You can achieve that by dynamically concatinating your Query with current date, or perform more advanced date function operations if necessary:您可以通过将 Query 与当前日期动态连接来实现这一点,或者在必要时执行更高级的日期函数操作:
@concat('SELECT FROM schema.table t WHERE reporting_date =', formatDateTime(utcnow(), 'yyyy-MM-dd'))
You can find more detailed list of whats possible in the following url: https://docs.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions#date-functions您可以在以下网址中找到更详细的可能性列表: https : //docs.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions#date-functions
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.