让基于日志的指标自动确定来自 JSON 过滤日志有效负载的标签

Question

Background (skip to "The Problem" for end goal)

Take an example endpoint /process-batch that processes a batch of items given a single request. Upon receiving that request, it logs the following line:

{"endpoint_type": "process-batch", "batch_size": <integer>, "are_there_warnings": <boolean>}

I can configure a logs-based metric for this via:

Then I can create a line chart (changing the "Resource & Metric" to be the logs-based metric created above, specifying that it can be emitted from anywhere globally):

Resulting dashboard looks like this:

The issue is that batch_size and are_there_warnings don't automatically appear as fields that I can use to manipulate the line graph. See "Group By" and "Add Filter" below:

To enable those fields, I can add a label for each of them (let's call this the "explicit labeling" step for future reference):

After confirming those labels are added:

I can go back to the dashboard and see that "Group By" and "Add Filter" have been updated accordingly:

And can manipulate the dashboard to use those fields:

The Problem

When trying to improve observability for an existing codebase, I have to add all the existing fields already logged as explicit labels on the GCP side, instead of GCP inferring that the logged fields are labels based on their name and type. And for the rare cases when I add a new field (such as cost to the log lines emitted by /process-batch ), I have to do explicit labeling on the GCP Monitoring and Logging side instead of GCP inferring that the newly added field should be a new implicitly added label in group by and filter.

Is it possible for GCP to automatically treat all fields part of jsonPayload as labels? Here are some reasonable defaults I had in mind:

• "Log metric name" takes on the name of the field itself.
• "Description" would be left empty.
• "Label type" would be determined from the type of the field.
• "Field name" would be jsonPayload.<log-metric-name> .

If this is not possible, I'd also appreciate any reasoning if implicitly treating jsonPayload fields as labels is a bad idea. In my eyes, it would significantly reduce the effort to have functional dashboards.

Answer 1

Upon checking, This kind of feature is not yet available. However we already raise this request. This request does not have an ETA but we can monitor this using below link.

You can star the public issue tracker feature requests and add 'Me too' in the thread. This will bring more attention to the request as more users request support for it.

Feature Request Tracker: Autodetect Labels from JSON Payload

As of now below are some guides that can help you.

Google Cloud fluentd output plugin configuration

Configuring the agent

Processing payloads

Special fields in structured payloads