[英]How to group logs messages in logstash using grok?
[25-Dec-2015 08:06:45] 0:: users to chek for delete
[25-Dec-2015 08:08:44] 0:: users to chek for delete
[25-Dec-2015 08:10:44] 3:: users to chek for delete
[25-Dec-2015 08:10:44] Expected response code 200, got 404
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "Resource Not Found: userKey"
}
],
"code": 404,
"message": "Resource Not Found: userKey"
}
}
[06-Nov-2015 19:24:19 GMT] PHP Fatal error: Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
[06-Nov-2015 19:24:19 GMT] PHP Stack trace:
[06-Nov-2015 19:24:19 GMT] PHP 1. {main}() /apps/test/public/api.php:0
[06-Nov-2015 19:24:19 GMT] PHP 2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
[06-Nov-2015 19:24:19 GMT] PHP 3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14
Above is sample of my log file. 上面是我的日志文件的示例。 I need to filter out each message.
我需要过滤掉每条消息。 the problem is with writing filters.
问题在于编写过滤器。 The first 3 lines are three different errors.
前三行是三个不同的错误。
The fourth error is a error with JSON message. 第四个错误是JSON消息错误。 I need to separate this block from above.
我需要将此块与上面分开。
[25-Dec-2015 08:10:44] Expected response code 200, got 404
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "Resource Not Found: userKey"
}
],
"code": 404,
"message": "Resource Not Found: userKey"
}
}
The fifth error is PHP stack trace. 第五个错误是PHP堆栈跟踪。
[06-Nov-2015 19:24:19 GMT] PHP Fatal error: Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
[06-Nov-2015 19:24:19 GMT] PHP Stack trace:
[06-Nov-2015 19:24:19 GMT] PHP 1. {main}() /apps/test/public/api.php:0
[06-Nov-2015 19:24:19 GMT] PHP 2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
[06-Nov-2015 19:24:19 GMT] PHP 3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14
Is it possible to design a grok filter to match these 3 conditions? 是否可以设计一个符合上述3个条件的过滤器?
Use the multiline
option. 使用
multiline
选项。 For example: 例如:
filter {
multiline {
negate => true
pattern => "^\["
what => "previous"
}
}
The result should look like this: 结果应如下所示:
[06-Nov-2015 19:24:19 GMT] PHP Fatal error: Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
PHP Stack trace:
PHP 1. {main}() /apps/test/public/api.php:0
PHP 2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
PHP 3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14
Your first step is to get the multi-line json error into one logstash event. 第一步是将多行json错误放入一个logstash事件中。 Check out the multiline codec or filter.
签出多行编解码器或过滤器。 Then, I would recommend using one grok{} stanza to pull the datetime off the line, and then use another grok stanza to process the remaining part of the line.
然后,我建议使用一个grok {}节将日期时间下线,然后使用另一个grok节处理该行的其余部分。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.