通过 Azure function 将数据摄取到 Azure 数据浏览器

Question

I am new to Azure and I want to know whether we can send some json data, process it and ingest it in data explorer. My requirement is to process the data on azure side before ingesting it to the data explorer . Can I use the following steps to do this process?

1. Trigger the azure function via HTTP request with json data
2. Process the data in the azure function
3. Ingest the process data via data explorer SDK

Is this possible or do I have to set some mapping with event hub or blob storage and use the their mapping properties with the data explorer. If that's the case, Is it possible to send data to event hub or blobstorage on http trigger azure functions?

Answer 1

There are multiple ways in which you can ingest data into Azure Data Explorer.

For the approach that you described, if the processing of JSON data will not take longer time, then you could do that inside Azure Function and use one of the SDK to ingest to data explorer. But Azure Function max timeout limit is 10 min in Consumption plan, and even in Premium plan it is not advisable to make it longer[though MS theoretical limit is unbounded ]. So, the above approach will work depending on your JSON data size, and 'processing time' and then to some extent the language in which you implement the Azure function.

Refer the official comparison between various methods using which you can ingest into Data Explorer. If you provide more details about your data size, and how long your 'processing' likely would take, it helps to make a better choice.