[英]Logstash Grok pattern with double quotes
I am parsing proxy logs with Logstash and its Grok filter. 我正在使用Logstash及其Grok过滤器解析代理日志。 The logs contain quoted strings :
日志包含带引号的字符串:
1438120705 [.....] "SEF-EDP8" - "C"
"/GPM/1023/5745-7/456V/"
With the Grok Debugger the following pattern works like a charm : 使用Grok Debugger ,以下模式就像魅力:
%{NUMBER:ts} [......] (-|"%{USERNAME:token1}") (-|%{DATA:token2}) (-|"%{WORD:token3}") (-|"%{DATA:token4}")
This does not work with Logstash's Grok because of the double quotes in the grok pattern. 这不适用于Logstash的Grok,因为grok模式中有双引号。 Logstash error log :
Logstash错误日志:
Error: Expected one of #, {, } at line 9, column 204 (byte 374) after
filter {
grok {
match => { "message" => "%{NUMBER:ts} [......] ("
So I use the QuotedString grok pattern instead : 所以我使用QuotedString grok模式代替:
%{NUMBER:ts} [......] (-|%{QS:token1}) (-|%{DATA:token2}) (-|%{QS:token3}) (-|%{QS:token4})
This works with the Grok Debugger as well, but quotes are extracted with quoted strings. 这也适用于Grok调试器 ,但引号是用带引号的字符串提取的。 It doesn't work with Logstash either :
它也不适用于Logstash :
token1 : ""SEF-EDP8"" token2 : null token3 : ""C"" token4 :
""/GPM/1023/5745-7/456V/""
How can I make it work with Logstash ? 如何使其与Logstash一起使用 ? How can I remove these unwanted extra double quotes ?
如何删除这些不需要的额外双引号 ?
If you escape " with backslash then it works fine. 如果你逃避“反斜杠,那么它工作正常。
%{NUMBER:ts} [......] (-|"%{USERNAME:token1}") (-|%{DATA:token2}) (-|"%{WORD:token3}") (-|"%{DATA:token4}") %{NUMBER:ts} [......]( - |“%{USERNAME:token1}”)( - |%{DATA:token2})( - |“%{WORD:token3}”)( - | “%{DATA:token4}”)
Your new string will look like 你的新字符串看起来像
%{NUMBER:ts} [......] (-|\\"%{USERNAME:token1}\\") (-|%{DATA:token2}) (-|\\"%{WORD:token3}") (-|\\"%{DATA:token4}\\") %{NUMBER:ts} [......]( - | \\“%{USERNAME:token1} \\”)( - |%{DATA:token2})( - | \\“%{WORD:token3}” )( - | \\“%{DATA:token4} \\”)
Changing the outer double quotes to single quotes instead did the trick for me: 将外部双引号更改为单引号代替了我的诀窍:
grok {
match => { "message" => 'SOME "TEXT QUOTED"' }
}
Hope it helps. 希望能帮助到你。
Try gsub after you have extracted the fields with quotes 使用引号提取字段后尝试gsub
filter {
mutate {
gsub => [
"fieldname", "\"", ""
]
}
}
https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.