How to get data from a SQL database into storage blob for reference data

Question

I have a azure SQL database and want to use the data from the database for Azure Stream Analytics - I know that I have to use a Blob Storage in order to use my data as reference data for my Stream Analytics Job.

Question

Now what would be the best way to get my SQL data into the Stream Analytics Job to enrich my incoming data?

Answer 1

You need to use the 'Azure data factory'. The 'data factory' can be used to orchestrate and automate the movement and transformation of your reference data from other cloud or on-premise data stores.

Check this article for a nice example on how to pull in data from a SQL Azure database to the Blob storage - Refreshing reference data

Some links on 'Azure data factory' -

• Introduction to Azure Data Factory
• Moving data using Azure Data Factory

Answer 2

Remarks to Previous Answer

Azure Data Factory is a whole new product and oversized, I think it's not appropriate for the relatively small size of what I want to do.

Answer

To get the SQL data into the Stream Analytics you have to save it in a Blob Storage (a simple WebApp is enough) and then it can be used to inside the Stream Analytics to enrich the data which is coming from the IoT Hub (eg appending a location to an object which is known to the front end but has no location in the front end due to data separation concerns).

In the Azure Blob Storage a snapshot from the current state of the SQL database is stored - this is okay for me because neither the amount nor the location of my objects changes very often, so updates to the Blob Storage are very infrequent.

In the Azure Stream Analytics a Blob Storage can be used to append information to a model, so here the model ids can be matched and a location can be appended, then the modified model can be saved again to a blob storage for a later data analysis.