Custom docker container in Kubernetes cluster with log using Stackdriver

Question

I would like to know which steps I have to follow in order to send the logs created in my custom apache container (deployed in a pod with Kubernetes) to the Stackdriver collector.

I have noticed that if I create a pod with a standard apache (or nginx) container, access.log and error.log are sent automatically to Stackdriver.

In fact I'm able to see the log both on Kubernetes dashboard and on Google Cloud Dashboard--->Logging--->Logs Instead I don't see anything related my custom apache...

Any suggestions?

Answer 1

After some researches I have resolved the problem of log forwarder from my custom apache container.

I don't know why the "standard redirection" (using /dev/stdout or /proc/self/fd/1) is not working anyway the solution that I followed is called "sidecar container with the logging agent"

1) create a configMag file where you'll set a fluentd configuration:

apiVersion: v1
data:
  fluentd.conf: |
    <source>
      type tail
      format none
      path /var/log/access.log
      pos_file /var/log/access.log.pos
      tag count.format1
    </source>

    <source>
      type tail
      format none
      path /var/log/error.log
      pos_file /var/log/error.log.pos
      tag count.format2
    </source>

    <match **>
      type google_cloud
    </match>
kind: ConfigMap
metadata:
  name: my-fluentd-config

2) create a pod with 2 containers: the custom apache + a log agent. Both containers will mount a log folder. Only log agent will mount the fluentd config:

apiVersion: v1
kind: Pod
metadata:
  name: my-sidecar
  labels:
    app: my-sidecar 
spec:
  volumes:
  - name: varlog
    emptyDir: {}
  - name: config-volume
    configMap:
      name: my-fluentd-config 

  containers:
  - name: my-apache
    image: <your_custom_image_repository>
    ports:
      - containerPort: 80
        name: http
        protocol: TCP 
    volumeMounts:
    - name: varlog
      mountPath: /var/log

  - name: log-agent
    image: gcr.io/google_containers/fluentd-gcp:1.30
    env:
    - name: FLUENTD_ARGS
      value: -c /etc/fluentd-config/fluentd.conf
    volumeMounts:
    - name: varlog
      mountPath: /var/log
    - name: config-volume
      mountPath: /etc/fluentd-config

3) Enter in my-apache container with:

kubectl exec -it my-sidecar --container my-apache -- /bin/bash

and change/check the httpd.conf is using the following files:

ErrorLog /var/log/error.log

CustomLog /var/log/access.log common

(if you change something remember to restart apache..)

4) Now in Google Cloud Console -> Logging you'll be able to see the apache access/error logs in Stackdriver with a filter like:

resource.type="container"
labels."compute.googleapis.com/resource_name"="my-sidecar"