[英]Using logstash to strip XSSI prefix from JSON response
I have a fairly simple problem but it's confusing to me. 我有一个很简单的问题,但这使我感到困惑。 I'm trying to use Logstash to get Gerrit data via rest api. 我正在尝试使用Logstash通过rest api获取Gerrit数据。 I'm using http_poller and I get a right response with my configuration, so I'm almost there. 我正在使用http_poller,并且我的配置得到了正确的响应,所以我快到了。
Now I need to strip the XSSI prefix )]}' from the start of Gerrits JSON response. 现在,我需要从Gerrits JSON响应的开头剥离XSSI前缀)]}' 。 The question is, how? 问题是,如何? How to strip or split or mutate it, or how should I proceed? 如何剥离,分割或变异它,或者应该如何进行?
My input configuration: 我的输入配置:
input {
http_poller {
urls => {
gerrit_projects => {
method => get
url => "http://url.to/gerrit/a/projects/"
headers => { Accept => "application/json" }
auth => { user => "userid" password => "supresecret" }
}
}
target => "http_poller_data"
metadata_target => "http_poller_metadata"
request_timeout => 60
interval => 60
}
}
filter {
if [http_poller_metadata] {
mutate {
add_field => {
"http_poller_host" => "%{http_poller_metadata[host]}"
"http_poller" => "%{http_poller_metadata[name]}"
}
}
}
if [http_poller_metadata][runtime_seconds] and [http_poller_metadata][runtime_seconds] > 0.5 {
mutate { add_tag => "slow_request" }
}
if [http_request_failure] or [http_poller_metadata][code] != 200 {
mutate { add_tag => "bad_request" }
}
}
output {
stdout { codec => rubydebug }
}
And parts of the response: 以及部分响应:
Pipeline main started
JSON parse failure. Falling back to plain-text {:error=>#<LogStash::Json::ParserError: Unexpected character (')' (code 41)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at ... (bunch of lines)...
{
"http_poller_data" => {
"message" => ")]}'\n{\"All-Users\":{\"id\":\"All-Users\",....(more valid JSON)...",
"tags" => [
[0] "_jsonparsefailure"
],
"@version" => "1",
"@timestamp" => "2016-12-13T09:48:25.397Z"
},
"@version" => "1",
"@timestamp" => "2016-12-13T09:48:25.397Z",
"http_poller_metadata" => { ... }
This is my first question to StackOverflow. 这是我对StackOverflow的第一个问题。 Thank you for being kind with your answers! 多谢您的回答!
I use "sed 1d" to remove the ")]}'" prefix and "jq" to process the JSON output. 我使用“ sed 1d”删除“)]}'”前缀,并使用“ jq”处理JSON输出。 For example, to get the state of a Gerrit project I execute: 例如,要获取Gerrit项目的状态,请执行:
curl -s --header 'Content-Type:application/json' --request GET --netrc https://<GERRIT-SERVER>/a/projects/?r=<GERRIT-PROJECT> | sed 1d | jq --raw-output ".[] | .state"
ACTIVE
You can use the mutate filter with the gsub option ( link ) to remove the )]}
您可以将mutate过滤器与gsub选项( link )一起使用以删除)]}
mutate {
gsub => [
"message", "\)]}'", ""
]
}
But the gsub replace all occurences of a regex, so you have to be sure that the pattern only appears once. 但是gsub替换了所有出现的正则表达式,因此您必须确保该模式仅出现一次。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.