Docker daemon keeps writing disk and cause shell commands to run slowly

Question

My sandbox computer is very very slow recently, and after digging into it I found the docker daemon is writing to disk frequently as below

 TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND                                                                
25321 be/4 root        0.00 B/s  297.06 K/s  0.00 %  0.00 % dockerd -H fd:// --containerd=/run/containerd/containerd.sock
25344 be/4 root        0.00 B/s  246.24 K/s  0.00 %  0.00 % dockerd -H fd:// --containerd=/run/containerd/containerd.sock
25351 be/4 root        0.00 B/s  148.53 K/s  0.00 %  0.00 % dockerd -H fd:// --containerd=/run/containerd/containerd.sock
25352 be/4 root        0.00 B/s  328.32 K/s  0.00 %  0.00 % dockerd -H fd:// --containerd=/run/containerd/containerd.sock
25514 be/4 root        0.00 B/s  555.03 K/s  0.00 %  0.00 % dockerd -H fd:// --containerd=/run/containerd/containerd.sock
25536 be/4 root        0.00 B/s  343.96 K/s  0.00 %  0.00 % dockerd -H fd:// --containerd=/run/containerd/containerd.sock

I tried to stop the docker, and everything comes normal again. But when I restart the docker, it turns into the slow state, shell commands show up the outputs slowly.

Why would docker daemon keeps writing disk? how to prevent it from writing disk?

My docker version is 19.03.

Answer 1

To find out which files these Docker processes are writing to, you could use the strace comand:

strace -e trace=file -p <PID>

The process ID (PID) is the same as the "TID" in your iotop output.