Is it possible to log Azure Cloud Service Performance Counters in Elasticsearch?

Question

I'd like to use Kibana to create views that display log and metric information output by our Azure Cloud Service Web Roles and Worker Roles. In particular, we'd like to store performance counter information, as described here: https://docs.microsoft.com/en-us/azure/cloud-services/cloud-services-dotnet-diagnostics-performance-counters

Microsoft provides a few tools to view this data if it's stored in Azure diagnostic tables and blobs, but the tools don't have the formatting & visualizing flexibility the Elk stack does. Is anyone aware of how we might be able to get Azure Cloud Service Performance Counter information into Elasticsearch, preferably via logstash?

Answer 1

I've achieved this by enabling Azure diagnostics (writing to Azure table storage) and using ConveyorBelt to send the logs to ElasticSearch.

As the GitHub page states, ConveyorBelt is:

A horizontally scalable headless cluster to shovel Azure diagnostic data (and other custom data) to ElasticSearch