Azure Data Factory - Azure SQL Database as JSON API to Power BI

Question

I want to us Azure Data Factory for Data Management and Link it with Power BI Desktop.

I am new to Azure and I don't have an in depth tech background, so this might be a quite trivial question. However, I plan to do the following in several stages and for Stage 1 I want:

STAGE 1

1.1 Connect 1 Azure DB that consists of one table to my Data Factory.

1.2 If possible represent this table as JSON and have an API that lets me connect with Power BI desktop.

Stage 2:

2.1 Connect multiple Azure DBs to ADF, transform and then

2.2 = 1.2 (do the same as above 1.2)

Stage 3 to Stage n: - I want to create everything as "flexible" as possible since I don't know where exactly this project will go.

I am also open to any suggestions that would do the trick better then how I picture it. Open for any suggestion on how to pull it off.

Many thanks in advance! I appreciate your time and effort

Best, MLU

Answer 1

I'm going to try to address some of your issues, but for most StackOverflow questions, you are probably going to need a bit more focused technical request.

First - there is no connection between Azure Data Factory (ADF) and Power BI (PBI). ADF is an ETL/ELT orchestration tool, so its primary function is to organize data operations.

Second - ADF connecting to databases (or other data sources, like Blob Storage) is a primary capability. This is done via a mechanism called Linked Services (found under the Connections section in the web UI). For your request, you would create such a connection to your SQL table and one to a destination (like Blob Storage) for the transformed JSON.

Third - there are several ways to perform such a transform. The easiest one to get started with is the Copy activity, which reads from a "Source" (your DB) and writes to a "Sink" (JSON in Blob Storage). In ADF, the definition of the source and sink data structure/format is known as a "DataSet". Much of the control for this kind of operation is defined by the underlying DataSets.

Fourth - Copy activity is limited in the amount of transformation it can perform. When you hit those boundaries, you'll want to investigate Azure Data Factory Data Flows (ADFDF). Data Flows provide a visual wrapper around Azure Data Bricks, and are well beyond the scope of this answer. Suffice it to say they give you far more capabilities to address complex problems.

PBI is a consumer - it reads data sources and does stuff with it. ADF is not a data source, hence no real connection between the two. As a data consumer, PBI can easily read your DB(s), so the real question is why would you want/need to convert it to JSON in the first place? It feels like you are going down an unnecessary rabbit hole: unless you have a particular need for a JSON transformation, I would bypass that altogether.