How to debug ingress-controller connections with a single IP by ConfigMap

Question

We are trying to edit our ingress-nginx.yml to make ingress-controllers pods debug traffic coming from a specific source IP . Our setup is:

• Kubernetes v1.13
• Ingress-Controller v0.24.1

From NGINX and Kubernetes DOCs it appears there is no very easy way to debug traffic from a single ip (you cannot edit the nginx config directly). So, we would like to add the debug_connection directive to appear like this:

error_log /path/to/log;
...
events {
    debug_connection 192.168.1.1;
}

The correct way to do it shall be through CustomAnnotations in a ConfigMap + a new ingress to enable the CustomAnnotation, so we tried this:

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
data:
ingress-template: |
    #Creating the custom annotation to make debug_connection on/off
    {if index $.Ingress.Annotations "custom.nginx.org/debug_connection"}
    {$ip := index $.Ingress.Annotations "custom.nginx.org/ip"}
    {end}

    {range $events := .Events}
    events {
      # handling custom.nginx.org/debug_connection
      {if index $.Ingress.Annotations "custom.nginx.org/debug_connection"}
      {end}

And:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: debugenabler
  annotations:
    kubernetes.io/ingress.class: "nginx"
    custom.nginx.org/debug_connection: "on"
    custom.nginx.org/ip: "192.168.1.1"
spec:
  rules:
  - host: "ourhostname"
    http:
      paths:
      - path: /tea
        backend:
          serviceName: tea-svc
          servicePort: 80
      - path: /coffee
        backend:
          serviceName: coffee-svc
          servicePort: 80

We applied ingress-nginx.yml with no errors. We see new lines in the nginx conf:

location /coffee {

            set $namespace      "test";
            set $ingress_name   "debugenabler";
            set $service_name   "coffee-svc";
            set $service_port   "80";
            set $location_path  "/coffee";

            rewrite_by_lua_block {
                lua_ingress.rewrite({
                    force_ssl_redirect = true,
                    use_port_in_redirects = false,
                })
                balancer.rewrite()

But still nothing as regard the debug_connection in the events block:

events {
    multi_accept        on;
    worker_connections  16384;
    use                 epoll;
}

How to insert debug_connection in the events context?

Answer 1

For those who may face similar challenges, I actually managed to do it by:

1. Creating a ConfigMap with a new ingress-controller template file ( nginx.tmpl ) containing the debug_connection line (double check your ingress-controller version here, the file changes dramatically)
2. Creating a Volume which links at the Configmap (specifying Volume and Volumemount)
3. Creating a InitContainer which copy the content of the volume inside the /etc/nginx/template (this was needed to overcome probably permission-related issues) before the container start.

For point 2 and 3 you can add the relevant code at the end of a deployment or a pod code, I share an example:

     volumes:
        - name: nginxconf2
          configMap:
            name: nginxconf2
            items:
            - key: nginx.tmpl
              path: nginx.tmpl       
      initContainers:
      - name: copy-configs
        image: {{ kubernetes.ingress_nginx.image }}
        volumeMounts:
        - mountPath: /nginx
          name: nginxconf2
        command: ['sh', '-c', 'cp -R /nginx/ /etc/nginx/template/']