简体   繁体   中英

fluentd parse log partially json

I have an source from syslog proto in fluentd, but the line is not fully syslog compliant.

142>May 31 16:22:56 haproxy[77]: {"ident":"haproxy","client_ip":"172.20.1.3","c....}

I need to parse the json part before submitting it to elasticsearch/kibana.

how would you do that? This is producing me a "failed to parse message data"

<source>
  @type syslog
  port 514
  tag haproxy-logs
  <parse>
    @type json
  </parse>
  key_name log
</source>

I also tried a standard syslog input, whithout parsing. and a filter whith json parser, but whitout success. thanks

So I got it working by doing it in 2 times:

<source>
  @type syslog
  port 514
  tag haproxy-logs
  <parse>
    @type regexp
    expression /(?<jsonmsg>\{\".*)/
  </parse>
</source>

and

<filter haproxy-logs**>
  @type parser
  format json
  time_key date_time
  time_format %d/%b/%Y:%H:%M:%S.
  key_name jsonmsg
  reserve_data true
</filter>

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM