TIMESTAMP GROK FAILURE

Question

some issue with grok timestamp pattern

2020-3-4 10:22:37 >> this will match with this pattern %{TIMESTAMP_ISO8601:my_time}

2020-3-4 0:2:37 >> this will fail with this pattern %{TIMESTAMP_ISO8601:my_time}

also tried to match the pattern by using this separate pattern like YEAR MONTH AND DAY but again it will break when it reaches time %{HOUR}:%{MINUTE}:%{SECOND} . Any idea ?

Answer 1

The issue is with how the minute pattern is defined in logstash: (?:[0-5][0-9]) . This pattern expects a two digit minute number, which breaks in your second case ( 2020-3-4 0:2:37 ).

I've changed the pattern to accept a one-digit number of minutes: (?:[0-5][0-9]|[0-9])

You can then use this custom pattern:

(?<my_time>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?(?:[0-5][0-9]|[0-9])(?::?%{SECOND})?%{ISO8601_TIMEZONE}?)

which is the TIMESTAMP_ISO8601 with MINUTE replaced by my pattern.