How to get container and image name when using fluentd for docker logging?

Question

I am using a fluentd container that mounts /var/lib/docker/containers directory and tails all of the logs for each container. The logs are stored in /var/lib/docker/containers/$container_id/$container_id-json.log. These do not contain the container name or the image name. I need to be able to add the container and image name in separate fields into every log coming from the log file.

The container name and image name are both stored in /var/lib/docker/container/$container_id/config.v2.json. I can't figure out how to get the image and name from that file and add it as a record to it's corresponding logs.

This was only my first thought and there may be a better way to do this, feel free to offer any advice.

Sidenote: I am intentionally not using fluentd as a docker logging driver because we previously used logspout in a similar manner. The current goal is to replace logspout with fluentd in a similar way. If the logging driver does provide this information in a much easier way then I would consider switching over later on.

Answer 1

I've not really used fluentd before, so apologies for a slightly abstract response here. But .. checking on http://docs.fluentd.org/ I guess you're probably using in_tail for the logs? From the example there, it looks like you'd probably want to get the path to the file into the input message, a la:

path /path/to/file
tag foo.*

which apparently tags events with foo.path.to.file

I'm thinking you could probably then use http://docs.fluentd.org/articles/filter_record_transformer with enable_ruby . From this, it looks like you could probably process the foo.path.to.file tag and use a little ruby to extract the container ID and then parse out then JSON file?

For example, testing with the following ruby file, say, foo.rb :

tag = 'foo.var.lib.docker.containers.ID.ID-json.log'
require 'json'; id = tag.split('.')[5]; puts JSON.parse(IO.read("/var/lib/docker/containers/#{id}/config.v2.json"))['image']

where config.v2.json was something like:

{"image":"foo"}

will print you

foo

Fluentd might already be including json for you, so maybe you could leave out the require 'json'; bit. Then, putting this in fluentd terms, perhaps you could use something like

<filter>
  enable_ruby
  <record>
    container ${tag.split('.')[5]}
    image ${id = tag.split('.')[5]; JSON.parse(IO.read("/var/lib/docker/containers/#{id}/config.v2.json"))['image']}
  </record>
</filter>

You'd want to production-ise this a bit, but perhaps it could work?

Answer 2

Thanks @boybinall. It works. Here is my code.

<filter raw.**>
  @type record_transformer
  enable_ruby
  <record>
    container ${id = tag.split('.')[5]; JSON.parse(IO.read("/var/lib/docker/containers/#{id}/config.v2.json"))["Name"][1..-1]}
    hostname "#{Socket.gethostname}"
  </record>
</filter>