如何使用 docker 配置 fluentd？

Question

I am using a rails application now. In my application config file, I added this:

config/application.rb

logger = Fluent::Logger::FluentLogger.new(nil, host: '127.0.0.1', port: '24224')

Then boot app in docker, saw these error logs:

$ docker-compose up

...
sidekiq_1 | E, [2015-11-09T02:45:44.857664 #1] ERROR -- : Failed to connect fluentd: Connection refused - connect(2) for "127.0.0.1" port 24224
sidekiq_1 | E, [2015-11-09T02:45:44.860403 #1] ERROR -- : Connection will be retried.
sidekiq_1 | E, [2015-11-09T02:45:44.862926 #1] ERROR -- : Failed to connect fluentd: Connection refused - connect(2) for "127.0.0.1" port 24224
sidekiq_1 | E, [2015-11-09T02:45:44.864514 #1] ERROR -- : Connection will be retried.
web_1     | E, [2015-11-09T02:45:48.512481 #1] ERROR -- : Failed to connect fluentd: Connection refused - connect(2) for "127.0.0.1" port 24224
web_1     | E, [2015-11-09T02:45:48.512785 #1] ERROR -- : Connection will be retried.
web_1     | E, [2015-11-09T02:45:48.515245 #1] ERROR -- : Failed to connect fluentd: Connection refused - connect(2) for "127.0.0.1" port 24224
web_1     | E, [2015-11-09T02:45:48.515571 #1] ERROR -- : Connection will be retried.
...

My docker machine's ip:

$ docker-machine ip myapp
=> 192.168.99.100

My docker ps:

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
b5e980e1aa61        myapp_web         "bundle exec rails s "   About an hour ago   Up 6 minutes        0.0.0.0:3000->3000/tcp   myapp_web_1
43365dfb403f        myapp_sidekiq     "bundle exec sidekiq "   About an hour ago   Up 6 minutes                                 myapp_sidekiq_1
9570ee727f04        mysql               "/entrypoint.sh mysql"   About an hour ago   Up 6 minutes        0.0.0.0:3306->3306/tcp   myapp_db_1
e24495920c58        redis               "/entrypoint.sh redis"   2 days ago          Up 6 minutes        0.0.0.0:6379->6379/tcp   myapp_redis_1

Why failed to connect fluentd about sidekiq and web images?

---

Edit

Add this to Dockerfile :

RUN mkdir -p /fluentd/etc

Add this to docker-compose.yml :

fluentd:
  image: fluent/fluentd:latest
  ports:
    - "24224:24224"

When boot app, log shows:

fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: reading config file path="/fluentd/etc/fluent.conf"
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: starting fluentd-0.12.16
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: gem 'fluentd' version '0.12.16'
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: adding match in @mainstream pattern="docker.**" type="file"
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: adding match in @mainstream pattern="**" type="file"
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: adding filter pattern="**" type="stdout"
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: adding source type="forward"
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: using configuration file: <ROOT>
fluentd_1 |   <source>
fluentd_1 |     @type forward
fluentd_1 |     @id input1
fluentd_1 |     @label @mainstream
fluentd_1 |     port 24224
fluentd_1 |   </source>
fluentd_1 |   <filter **>
fluentd_1 |     @type stdout
fluentd_1 |   </filter>
fluentd_1 |   <label @mainstream>
fluentd_1 |     <match docker.**>
fluentd_1 |       @type file
fluentd_1 |       @id output_docker1
fluentd_1 |       path /fluentd/log/docker.*.log
fluentd_1 |       symlink_path /fluentd/log/docker.log
fluentd_1 |       append true
fluentd_1 |       time_slice_format %Y%m%d
fluentd_1 |       time_slice_wait 1m
fluentd_1 |       time_format %Y%m%dT%H%M%S%z
fluentd_1 |       buffer_path /fluentd/log/docker.*.log
fluentd_1 |     </match>
fluentd_1 |     <match **>
fluentd_1 |       @type file
fluentd_1 |       @id output1
fluentd_1 |       path /fluentd/log/data.*.log
fluentd_1 |       symlink_path /fluentd/log/data.log
fluentd_1 |       append true
fluentd_1 |       time_slice_format %Y%m%d
fluentd_1 |       time_slice_wait 10m
fluentd_1 |       time_format %Y%m%dT%H%M%S%z
fluentd_1 |       buffer_path /fluentd/log/data.*.log
fluentd_1 |     </match>
fluentd_1 |   </label>
fluentd_1 | </ROOT>
fluentd_1 | 2015-11-09 09:51:07 +0000 [info]: listening fluent socket on 0.0.0.0:24224
sidekiq_1 | E, [2015-11-09T09:51:11.887833 #1] ERROR -- : Failed to connect fluentd: Connection refused - connect(2) for "127.0.0.1" port 24224
sidekiq_1 | E, [2015-11-09T09:51:11.888508 #1] ERROR -- : Connection will be retried.
sidekiq_1 | E, [2015-11-09T09:51:11.893544 #1] ERROR -- : Failed to connect fluentd: Connection refused - connect(2) for "127.0.0.1" port 24224
sidekiq_1 | E, [2015-11-09T09:51:11.893926 #1] ERROR -- : Connection will be retried.

It seems that ip 127.0.0.1 doesn't right. I am using a docker-machine now.

Answer 1

Make sure your docker-compose.yml has a fluentd section ( as this one ), with at least the port 24224 exposed (you don't have top map it unless you want to access it directly from the host)

Original answer (2015)

Then your rail docker-compose.yml sidekiq section should have a link section:

links:
  - fluentd

Updated answer (2021, after battlecook 's comment ) Since links is obsolete , use depends_on

depends_on:
  - fluentd

See " Difference between links and depends_on in docker_compose.yml ".
Example: " How to use Dockerfile to link main container to a db container? ".