How to audit commands run by user inside a container in K8s

Question

I want to audit commands that are being run by a user inside a running pod.

I know that kube-apiserver supports audit policies that allows you to log every request that is being done towards the API but as far as I know the API audit only records the exec command and not the inner commands run afterwards.

An approach that I thought is to have a sidecar container with auditbeat running but it's too heavy and the user might be able to kill it.

Answer 1

The container should run a single process inside. It is not recommended to run a command inside container exception for testing. Most of our image doesn't have any type of shell.

If you have to spawn a shell and run a command inside, Then you need to think about is it possible to run that outside container? If the main process is terminated but your shell commands are running in a container then k8s might not terminate that pod and recreate a new pod which might impact HA

Answer 2

There are some commercial products that allow to do this. Few weeks ago I did a PoC for one of them. The way it's implemented is that their product running as a pod (with 1 container inside) on the host level (host namespace / HostPID) and tracks usage of Docker daemon.