I am trying to ingest JSON metric data from HTTP-Endpoints using Telegraf's HTTP Input Plugin , and write it to a Postgresql database using the Postgresql Output plugin . This is working as expected.
However, some of the JSON-Responses are very long . Since the PostgreSQL Output plugin creates a column for every field, these tables become very long and cluttered with information I don't need.
For those endpoints where the data cannot be altered at the sender's side (For example: RabbitMQ's Premade API), is there a way to filter the JSON Response in Telegraf?
For example: I have the following JSON-Response:
{
"message_stats": {
"publish": 1,
"publish_details": {
"rate": 0.0
}
}
}
I am only interested in message_stats.publish_details.rate
. I do not want message_stats.publish
to appear in my database. How can I achieve that?
One of the way how to solve this issue is using json_query
parameter. In this parameter, you can use special syntax for filtering the response:
[[inputs.http]]
## One or more URLs from which to read formatted metrics
urls = [
"https://url-to-metrics-endpoint"
]
# next field replace default metrics name "http" to your custom name:
name_override = "custom_name"
# set true if you want to ignore self signed certificate validation:
insecure_skip_verify = true
data_format = "json"
timeout = "5s"
# Here you can filter your JSON using special syntax:
json_query = "message_stats.publish_details.rate"
This is easiest example. More details about syntax you can find here (for example how to work with arrays and a much more): https://github.com/tidwall/gjson#path-syntax
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.