How to group logs messages in logstash using grok?

Question

 [25-Dec-2015 08:06:45] 0:: users to chek for delete
 [25-Dec-2015 08:08:44] 0:: users to chek for delete
 [25-Dec-2015 08:10:44] 3:: users to chek for delete
 [25-Dec-2015 08:10:44] Expected response code 200, got 404

     {
         "error": {
          "errors": [
           {
            "domain": "global",
            "reason": "notFound",
            "message": "Resource Not Found: userKey"
           }
          ],
          "code": 404,
          "message": "Resource Not Found: userKey"
         }
        }

    [06-Nov-2015 19:24:19 GMT] PHP Fatal error:  Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
    [06-Nov-2015 19:24:19 GMT] PHP Stack trace:
    [06-Nov-2015 19:24:19 GMT] PHP   1. {main}() /apps/test/public/api.php:0
    [06-Nov-2015 19:24:19 GMT] PHP   2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
    [06-Nov-2015 19:24:19 GMT] PHP   3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14

Above is sample of my log file. I need to filter out each message. the problem is with writing filters. The first 3 lines are three different errors.

• [25-Dec-2015 08:06:45] 0:: users to chek for delete
• [25-Dec-2015 08:06:45] 0:: users to chek for delete
• [25-Dec-2015 08:06:45] 3:: users to chek for delete

The fourth error is a error with JSON message. I need to separate this block from above.

    [25-Dec-2015 08:10:44] Expected response code 200, got 404
     {
         "error": {
          "errors": [
           {
            "domain": "global",
            "reason": "notFound",
            "message": "Resource Not Found: userKey"
           }
          ],
          "code": 404,
          "message": "Resource Not Found: userKey"
         }
        }

The fifth error is PHP stack trace.

        [06-Nov-2015 19:24:19 GMT] PHP Fatal error:  Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
        [06-Nov-2015 19:24:19 GMT] PHP Stack trace:
        [06-Nov-2015 19:24:19 GMT] PHP   1. {main}() /apps/test/public/api.php:0
        [06-Nov-2015 19:24:19 GMT] PHP   2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
        [06-Nov-2015 19:24:19 GMT] PHP   3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14

Is it possible to design a grok filter to match these 3 conditions?

Answer 1

Use the multiline option. For example:

filter {
    multiline {
        negate    => true
        pattern   => "^\["
        what      => "previous"
    }
}

The result should look like this:

[06-Nov-2015 19:24:19 GMT] PHP Fatal error:  Class 'Test\Test\Api\Resources\Authenticate1234' not found in /apps/test/src/Test/Test/Api/Resources/ResourceFactory.php on line 10
PHP Stack trace:
PHP   1. {main}() /apps/test/public/api.php:0
PHP   2. Test\Test\Api\ApiController->handleRequest() /apps/test/public/api.php:13
PHP   3. Test\Test\Api\Resources\ResourceFactory->create() /apps/test/src/Test/Test/Api/ApiController.php:14

Answer 2

Your first step is to get the multi-line json error into one logstash event. Check out the multiline codec or filter. Then, I would recommend using one grok{} stanza to pull the datetime off the line, and then use another grok stanza to process the remaining part of the line.