Grok Pattern optional fields on line end

Question

I want to "grok" these two lines:

usg210 CEF:0|ZyXEL|USG210|4.65(AAPI.1)|0|Access Control|5|devID=bccf4fxxxxxx src=192.168.1.228 dst=255.255.255.255 spt=7303 dpt=7303 msg=Match default rule, DROP proto=17 app=others

usg210 CEF:0|ZyXEL|USG210||0|Blocked Web Sites|9|devID=bccf4fxxxxxx src=192.168.1.228 dst=23.57.22.128 spt=50938 dpt=443 msg=gameplay.intel.com : Games, Rule_id=5, SSI=N (HTTPS Domain Filter)

Pattern:

\|(?:.*)\|%{DATA:class}\|%{WORD:loglevel}\|devID=%{WORD:mac} src=%{IPV4:ipsrc} dst=%{IPV4:ipdst} spt=%{WORD:spt} dpt=%{WORD:dpt} msg=%{GREEDYDATA:msg}( proto=%{WORD:proto} app=%{WORD:app})?

The fields 'proto' and 'app' are optional, but my debugger msg looks like this: "Match default rule, DROP proto=17 app=others" and proto an app is null.

Can anyone explain how to get the fields filled if data is present and how I make them optional if not present.

Answer 1

You are using GREEDYDATA pattern and do not require your pattern to match the whole string.

In Grok, you need to replace GREEDYDATA with DATA and add $ at the end of the pattern:

\|.*\|%{DATA:class}\|%{WORD:loglevel}\|devID=%{WORD:mac} src=%{IPV4:ipsrc} dst=%{IPV4:ipdst} spt=%{WORD:spt} dpt=%{WORD:dpt} msg=%{DATA:msg}( proto=%{WORD:proto} app=%{WORD:app})?$

See the msg=%{DATA:msg}( proto=%{WORD:proto} app=%{WORD:app})?$ part, with msg=%{DATA:msg} and $ .