APIM requests and Application Insights count

Question

We are using APIM for all our API requests and enabled Application Insights to make sure we get all information like country, request body, IP address, HTTP status code, etc.

We are using AppInsights API to get data APIM data, as on UI, there is a limit of 10K per query. https://api.applicationinsights.io/v1/apps/

It was working fine till we had a limited number of calls on APIM like 7K/10K per day.

Now we are getting around 40k-80K data per day.

Now when I write a Kusto query in the AppInsights UI, it give me counts 38,648, 29,493, 26,847 for 3 days.

requests
|where url contains 'abc'
|where timestamp >= startofday(datetime('30-Apr-20')) and timestamp <= endofday(datetime('02-May-20'))
| summarize count(), 
    avg(duration) 
      by bin(timestamp , 1d) 

But when I run an API query request, it gives me records around 54K whereas i should get records around 94K. When it runs for days where requests are more 150+, still it gives records around 54K.

I checked with the limit on the number of queries. they are talking about 200 per second 30 seconds and 86400 per day. Nothing is mentioned about data size.

It seems, there is a limitation on data size from AppInsights API When I download for 30-Apr to 01-May, file download size is around 74K When I download for 30-Apr to 02-May, still file download size is around 74K

I have used AppInsights API in C# console application and using webClient.(DownloadString/DownloadFIle) method to get this data. Query as follows

https://api.applicationinsights.io/v1/apps/<code/query?query=requests|where url contains 'abc'|where timestamp >= startofday(datetime('30-Apr-20'))and timestamp <= endofday(datetime('02-May-20'))

Answer 1

You have to set the sampling value to '100'.

How to integrate Azure API Management with Azure Application Insights

Sampling (%) decimal Values from 0 to 100 (percent). Specifies what percentage of requests will be logged to Azure Application Insights. 0% sampling means zero requests logged, while 100% sampling means all requests logged. This setting is used for reducing performance implications of logging requests to Azure Application Insights (see the section below).