unknown field "capabilities" in io.k8s.api.core.v1.PodSecurityContext (running tshark in a container/k8s pod)

Question

I have build a docker image containing tshark (its an image I am going to use for doing various manual debugging from a kubernetes pod).

I have deployed a container in kubernetes running that image. But when I access the container and try to run tshark I get:

$ kubectl exec myapp-cbd49f587-w2swx -it bash
root@myapp-cbd49f587-w2swx:/# tshark -ni any -f "test.host" -w sample.pcap -F libpcap
Running as user "root" and group "root". This could be dangerous.
Capturing on 'any'
tshark: cap_set_proc() fail return: Operation not permitted

Googling that error:

https://www.weave.works/blog/container-capabilities-kubernetes/ https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/

it seems I need to configure a securityContext for my container/pod. In my deployment.yaml I have added:

  containers:
     ...
  securityContext:
    capabilities:
      add:
        - NET_ADMIN

But when I apply that deployment I get:

error: error validating "deployment.yaml": error validating data: ValidationError(Deployment.spec.template.spec.securityContext): unknown field "capabilities" in io.k8s.api.core.v1.PodSecurityContext; if you choose to ignore these errors, turn validation off with --validate=false

Adding --validate=false removes the error but also means the securityContext is ignored.

What is preventing me from setting:

  securityContext:
    capabilities:
      add:
        - NET_ADMIN

Based on the guides I have found this should be fine.

I have also looked at (looks to be non free):

https://sysdig.com/blog/tracing-in-kubernetes-kubectl-capture-plugin/

so probably the right way is to use some tool like that ( ksniff ) or setup a sidecar container . But I am still curious to why I get the above error.

Answer 1

Looking specifically to the error, you posted only part of your manifest and looking to this we can see that you put securityContext: in the same level as containers: :

  containers:
     ...
  securityContext:
    capabilities:
      add:
        - NET_ADMIN

It should be under containers: as as written in the documentation :

To add or remove Linux capabilities for a Container, include the capabilities field in the securityContext section of the Container manifest.

Example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: security-context-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: security-context-demo
  template:
    metadata:
      labels:
        app: security-context-demo
    spec:
      containers:
      - name: sec-ctx-4
        image: gcr.io/google-samples/node-hello:1.0
        securityContext:
          capabilities:
            add:
            - NET_ADMIN

Answer 2

Linux capabilities can be added only at container level security-context, not at the pod level.

Not obvious here but see that the section on adding capabilities only mentions adding it to the container:

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-capabilities-for-a-container