How to send CosmosDB dependency tracking data to Application Insights from Azure Function Event Hub Trigger

Question

I have an Azure Function that is triggered by eventhub and sends data in batches. Inside function, there are multiple calls to insert data into CosmosDB. I have added following code as part of App Insight Monitoring.

 builder.Services.AddApplicationInsightsTelemetry();

            builder.Services.ConfigureTelemetryModule<DependencyTrackingTelemetryModule>((module, o) =>
            {
                module.EnableW3CHeadersInjection = true;
            });

         
            builder.Services.ConfigureTelemetryModule<EventCounterCollectionModule>(
                    (module, o) =>
                    {
                        module.Counters.Add(new EventCounterCollectionRequest("System.Runtime", "gen-0-size"));
                    }
                );

I can see total response time in App Insight but cannot figure out how to track and send time spent by each insert query in CosmosDB

Here is C# code within Azure Function

var watch = System.Diagnostics.Stopwatch.StartNew();
                    
                    var DemoContainerData = new
                       {
                           id = Guid.NewGuid().ToString(),
                           UserId = userId,
                           // other properties 
                       };

                       _demoContainer.CreateItemAsync<object>(DemoContainerData);
                       
                       var DemoContainerData2 = new
                       {
                           id = Guid.NewGuid().ToString(),
                           ProductId = productId,
                           // other properties 
                       };

                       _productContainer.CreateItemAsync<object>(DemoContainerData2);
              /* var dependency = new DependencyTelemetry
               {
                   Name = "",
                   Target = "",
                   Data = ",
                   Timestamp = start,
                   Duration = DateTime.UtcNow - start,
                   Success = true
               };
               this._telemetryClient.TrackDependency(dependency);
               */                    
            watch.Stop();
            var elapsed = watch.Elapsed.TotalMilliseconds;
            log.LogInformation("Total Items {0} - Total Time {1}", Items.Length, elapsed);     

Answer 1

Your code is not awaiting the async operations, you should be:

ItemResponse<object> response = await _demoContainer.CreateItemAsync<object>(DemoContainerData);

From the response, you can measure the client latency:

var elapsedTimeForOperation = response.Diagnostics.GetClientElapsedTime();

What we recommend if you want to investigate high latency is to log the Diagnostics when the request goes above some threshold, for example:

if (response.Diagnostics.GetClientElapsedTime() > ConfigurableSlowRequestTimeSpan)
    {
        // Log the diagnostics and add any additional info necessary to correlate to other logs 
        log.LogWarning("Slow request {0}", response.Diagnostics);
    }

For best latency, make sure you are following https://learn.microsoft.com/azure/cosmos-db/sql/troubleshoot-dot.net-sdk-slow-request?tabs=cpu-new#application-design (mainly make sure you are using a Singleton client, use ApplicationRegion or ApplicationPreferredRegions to define preferred region to connect which hopefully is the same region the Function is deployed to).