How to query Azure databricks data on Azure datafactory?

Question

My source data is in ADLS and its table format is in ADB data.

Now I'm creating a lookup activity inside the ADF pipeline where I have to write a SELECT query on the table present in ADB.

Can you please share your suggestions on how this link can be achieved?

Answer 1

The query property in the lookup activity accepts an expression, not a SQL statement.

Ref: https://docs.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions

You can set the source as a Stored Procedure or you can use the Copy activity to query the results from the result of the lookup.

See the example here: https://docs.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity#pipeline

Answer 2

if I understand the ask here you are trying to connect to an ADB instance and exeecute an query. Lookup activity does not support ADB. https://docs.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity AFAIK, you dont need to go the ADB route, you can directly query ADLS.