用Fluentd处理嵌套的JSON

Question

Using Fluentd log pipeline, with input of:

{"event" : {"name" : "toto", "date" : 14....11}}

How can I transform the field event.date to a valid Date?

I tried:

<filter *.*> @type record_transformer <record> event.date ${Time.at(event.date).strftime('%Y-%m-%dT%H:%M:%S')} </record> </filter>

But this adds a field "event.date" :/

Answer 1

It`s been a while, but I come here with a slight idea of how to do this. It seems that in newer versions, some JSON processing has become part of the core of fluentd, and according to the docs:

https://docs.fluentd.org/v1.0/articles/filter_parser

this here:

<filter foo.bar>
  @type parser
  key_name log
  reserve_data true
  remove_key_name_field true
  <parse>
    @type json
  </parse>
</filter>

should do the trick.

we have:

• key_name - the name of the field in which to look
• reserve_data - wether or not to keep the fields from the outer JSON
• remove_key_name_field - wether or not to remove the field containing the nested JSON.

Answer 2

You must enable rubby :

<filter **>
  @type record_transformer
  enable_ruby
  <record>
    event_date ${time.strftime('%Y-%m-%dT%H:%M:%S')}

  </record>

</filter>