Is there a way to syslog from container to underlying k8s node?

Question

I want to syslog from a container to the host Node -

Targeting fluentd (@127.0.0.1:5140) which runs on the node - https://docs.fluentd.org/input/syslog

eg syslog from hello-server to the node (which hosts all of these namespaces)

I want to syslog output from hello-server container to fluentd running on node (@127.0.0.1:5140).

kubectl get pods --all-namespaces
NAMESPACE     NAME                                             READY   STATUS    RESTARTS   AGE
default       hello-server-7d8589854c-r4xfr                    1/1     Running   0          21h
kube-system   event-exporter-v0.2.4-5f7d5d7dd4-lgzg5           2/2     Running   0          6d6h
kube-system   fluentd-gcp-scaler-7b895cbc89-bnb4z              1/1     Running   0          6d6h
kube-system   fluentd-gcp-v3.2.0-4qcbs                         2/2     Running   0          6d6h
kube-system   fluentd-gcp-v3.2.0-jxnbn                         2/2     Running   0          6d6h
kube-system   fluentd-gcp-v3.2.0-k58x6                         2/2     Running   0          6d6h
kube-system   heapster-v1.6.0-beta.1-7778b45899-t8rz9          3/3     Running   0          6d6h
kube-system   kube-dns-autoscaler-76fcd5f658-7hkgn             1/1     Running   0          6d6h
kube-system   kube-dns-b46cc9485-279ws                         4/4     Running   0          6d6h
kube-system   kube-dns-b46cc9485-fbrm2                         4/4     Running   0          6d6h
kube-system   kube-proxy-gke-test-default-pool-040c0485-7zzj   1/1     Running   0          6d6h
kube-system   kube-proxy-gke-test-default-pool-040c0485-ln02   1/1     Running   0          6d6h
kube-system   kube-proxy-gke-test-default-pool-040c0485-w6kq   1/1     Running   0          6d6h
kube-system   l7-default-backend-6f8697844f-bxn4z              1/1     Running   0          6d6h
kube-system   metrics-server-v0.3.1-5b4d6d8d98-k7tz9           2/2     Running   0          6d6h
kube-system   prometheus-to-sd-2g7jc                           1/1     Running   0          6d6h
kube-system   prometheus-to-sd-dck2n                           1/1     Running   0          6d6h
kube-system   prometheus-to-sd-hsc69                           1/1     Running   0          6d6h

For some reason k8s does not allow us to use the built in syslog driver docker run --log-driver syslog .

Also, k8s does not allow me to connect with the underlying host using --network="host"

Has anyone tried anything similar? Maybe it would be easier to syslog remotely rather than trying to use the underlying syslog running on every node?

Answer 1

What you are actually looking at is the Stackdriver Logging Agent . According to the documentation at https://kubernetes.io/docs/tasks/debug-application-cluster/logging-stackdriver/#prerequisites :

If you're using GKE and Stackdriver Logging is enabled in your cluster, you cannot change its configuration, because it's managed and supported by GKE. However, you can disable the default integration and deploy your own.

The documentation then gives an example of rinning your own fluentd DaemonSet with custom ConfigMap. You'd need to run your own fluentd so you could configure a syslog input per https://docs.fluentd.org/input/syslog .

Then, since the fluentd is running as a DaemonSet, you would configure a Service to expose it to other pods and allow then to connect to it. If you are running the official upstream DaemonSet from https://github.com/fluent/fluentd-kubernetes-daemonset then a service might look like:

apiVersion: v1
kind: Service
namespace: kube-system
metadata:
  name: fluentd
spec:
  selector:
    k8s-app: fluentd-logging
  ports:
  - protocol: UDP
    port: 5140
    targetPort: 5140

Then your applications can log to fluentd.kube-system:5140 (see using DNS at https://kubernetes.io/docs/concepts/services-networking/service/#dns ).