[英]Regexp JSON filtering in LogQL
I'd like to translate Kibana query like to following to LogQL:我想将 Kibana 查询转换为 LogQL:
host:("test1-myservice-*") AND level:ERROR
AND NOT logger_name:"com.example.ExampleClass"
AND _exists_:stack_trace
AND NOT stack_trace:(
"interrupted"
OR "Read timed out"
OR "java.lang.InterruptedException"
)
I have tried the following in Grafana Explore but it does not return any records for our JSON log messages:我在 Grafana Explore 中尝试了以下操作,但它没有为我们的 JSON 日志消息返回任何记录:
{host=~"test1-myservice-.*"} | json
| logger_name != "com.example.ExampleClass"
| stack_trace !=""
| stack_trace =~ ".*InterruptedException.*"
While using !=
instead of =~
it returns all records:使用
!=
而不是=~
它返回所有记录:
{host=~"test1-myservice-.*"} | json
| logger_name != "com.example.ExampleClass"
| stack_trace !=""
| stack_trace !~ ".*InterruptedException.*"
If I'm right the following applies from the documentations to the stack_trace
field of the JSON log line:如果我是对的,以下内容适用于文档中的 JSON 日志行的
stack_trace
字段:
String type work exactly like Prometheus label matchers use in log stream selector.
字符串类型的工作方式与日志 stream 选择器中使用的 Prometheus label 匹配器完全相同。 This means you can use the same operations (=,,=,=~.!~).
这意味着您可以使用相同的操作 (=,,=,=~.!~)。
Source: Label filter expression来源: Label滤波器表达式
The following seems to work but it seems awkward:以下似乎有效,但似乎很尴尬:
{host=~"test1-myservice-.*"} | json
| logger_name != "com.example.ExampleClass"
| stack_trace !=""
!~ ".*InterruptedException.*|.*Read timed out.*"
| json
Furthermore, if I'm right, it searches for InterruptedException
and Read timed out
substrings in the complete JSON string instead of its stack_trace
field only.此外,如果我是对的,它会在完整的 JSON 字符串中搜索
InterruptedException
和Read timed out
子字符串,而不仅仅是它的stack_trace
字段。
Is there a more LogQL-ish way to translate the Kibana query above to LogQL?是否有更类似于 LogQL 的方式将上面的 Kibana 查询转换为 LogQL? Should the
!~
operator work in this case? !~
运算符应该在这种情况下工作吗?
Environment: Grafana 7.5.4 / 8.2.3, Loki: 2.4.1环境:Grafana 7.5.4 / 8.2.3,Loki:2.4.1
Not sure how your log lines look exactly, but I think you don't need to extract the labels out (by using | json
不确定您的日志行看起来如何,但我认为您不需要提取标签(通过使用
| json
This is a pretty useful article on how to write queries.这是一篇关于如何编写查询的非常有用的文章。 how-to-create-fast-queries-with-lokis-logql-to-filter-terabytes-of-logs-in-seconds You can also make use of the new Pattern parser instead of the regex if you want to make the query more readable.
how-to-create-fast-queries-with-lokis-logql-to-filter-terabytes-of-logs-in-seconds如果要进行查询,您还可以使用新的模式解析器而不是正则表达式更具可读性。
So without really knowing how your log lines look, I think this should work well:所以在不知道你的日志行看起来如何的情况下,我认为这应该很好用:
{host=~"test1-myservice-.*"}
!= "com.example.ExampleClass"
!~ ".*InterruptedException.*|.*Read timed out.*"
Based on your needs you can also make use of the Pattern parser I've mentioned before.根据您的需要,您还可以使用我之前提到的模式解析器。
This works:这有效:
{host=~"test1-myservice-.*"} | json
| logger_name != "com.example.ExampleClass"
| stack_trace !=""
| stack_trace !~ "(?s).*InterruptedException.*"
Note the (?s)
which enables matching new lines for the regex .
请注意
(?s)
,它可以为 regex 匹配新行.
character.特点。 (The
stack_trace
field of the JSON log message usually contain multiple lines.) (JSON 日志消息的
stack_trace
字段通常包含多行。)
This is also mentioned in the Log stream selector part of the documentation :这也在文档的日志 stream 选择器部分中提到:
Note: The
=~
regex operator is fully anchored, meaning regex must match against the entire string, including newlines.注意:
=~
正则表达式运算符是完全锚定的,这意味着正则表达式必须匹配整个字符串,包括换行符。 The regex.
正则表达式
.
character does not match newlines by default.默认情况下,字符不匹配换行符。 If you want the regex dot character to match newlines you can use the single-line flag, like so:
(?s)search_term.+
matchessearch_term\n
.如果您希望正则表达式点字符匹配换行符,您可以使用单行标志,如下所示:
(?s)search_term.+
匹配search_term\n
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.