如何在 Azure 数据工厂中为雪花连接执行下推优化

Question

Recently Microsoft launched the snowflake connection for data flow in ADF. Is there any way to turn on the push down optimization in ADF so that if my source and target is Snowflake only then instead of pulling data out of snowflake environment it should trigger a query in snowflake to do the task. Like a normal ELT process instead of ETL.

Let me know if you need some more clarification.

Answer 1

As I understand the intent here is to fire a query from ADF on snowflake data so that possibley the data can be scrubbed ( or something similar ) . I see that Lookup activity also supports snowflake and probably that should help you . My knowledge on SF is limited , but i know that you can call a proc/query from lookup activity and that should help .

https://docs.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity

"Lookup activity reads and returns the content of a configuration file or table. It also returns the result of executing a query or stored procedure. The output from Lookup activity can be used in a subsequent copy or transformation activity if it's a singleton value. The output can be used in a ForEach activity if it's an array of attributes."