Which grok pattern for logstash multiline with different event dividers

Question

I have a log file (zope/plone event.log) which using custom string (eg "-----") as divider between events, how grok pattern for parsing this log file to logstash should be?

This is an example how the log look like:

------
2014-07-21T12:13:30 INFO ZServer HTTP server started at Mon Jul 21 12:13:30 2014
        Hostname: localhost
        Port: 8401
------
2014-07-21T12:13:44 WARNING SecurityInfo Conflicting security declarations for "setText"
------
2014-07-21T12:13:44 WARNING SecurityInfo Class "ATTopic" had conflicting security declarations
------
2014-07-21T12:13:47 INFO DocFinderTab Applied patch version 1.0.5.

Answer 1

You should start with the multiline codec or filter to create a single event for processing.

EDIT:

The doc gives this example:

filter {
  multiline {
    pattern => "pattern, a regexp"
    negate => boolean
    what => "previous" or "next"
  }
}

And describes what 'negate' and 'what' do. Hopefully 'pattern' make sense.

So, how about "every line that doesn't start with a date belongs with the prior line"? That might be something like this:

filter {
  multiline {
    negate => 'true'
    pattern => "^%{TIMESTAMP_ISO8601} "
    what => 'previous'
  }
}

You'd be left with the "----" at the end of each line. Since you don't need them as delimiters, you can get rid of them (before the multiline filter stanza):

if message =~ /^-+$/ {
    drop{}
}