Why I cannot kill python3 process in k8s pod?

Question

I tried to kill one python process:

# ps aux | grep python
root           1 12.6  2.1 2234740 1332316 ?     Ssl  20:04  19:36 /usr/bin/python3 /batch/run.py
root         490  0.0  0.0  11472  1012 pts/0    S+   22:39   0:00 grep --color=auto python
# sudo kill -9 1
# ps aux | grep python
root           1 12.6  2.1 2234740 1333372 ?     Ssl  20:04  19:38 /usr/bin/python3 /batch/run.py
root         494  0.0  0.0  11472  1088 pts/0    S+   22:39   0:00 grep --color=auto python

Any idea why? Thanks. Any other information needed to debug this?

UPDATE

In fact, I would not like to kill the container or pod. I modified the python codes in /usr/local/lib/python3.6/dist-packages/ directly. If pod restarted, my changes will be gone.

I need to modify third party codes (not my own codes) in the containers and see the results directly.

Updating my own codes and redeploying the docker image is not my first choice in fact. Otherwise, why I ask questions here.

Also, I am curious why it cannot be killed?

Thanks

Answer 1

As mentioned by @coderanger, container has the concept to be immutable. What you trying to do isn't a good practice in Kubernetes/container environments.

But...

Sometimes a kind of magic is required keep the airplane in flight... There are some options that could help you:

1. Rebuild container image

The best solution in this case is rebuild your container image based in the current running image. You could run this image separately from your main workload to test the changes. That is the best approach in this case, because you'll persist the changes in the image and the historical for rolling updates.

2. Workaround to kill the pid

I've tested in a container running flask with supervisord.

You could use the SIGHUP signal to restart the process inside your container:

SIGHUP - The SIGHUP signal disconnects a process from the parent process. This an also be used to restart processes. For example, "killall -SIGHUP compiz" will restart Compiz. This is useful for daemons with memory leaks. ... SIGHUP P1990 Term Hangup detected on controlling terminal or death of controlling process

Inside your container, run:

kill -SIGHUP <PID> or kill -1 <PID>

Sources: - http://man7.org/linux/man-pages/man7/signal.7.html https://www.linux.org/threads/kill-signals-and-commands-revised.11625/

Answer 2

Containers are generally mostly immutable. Once started, you can't change the code that is running without very special handling for it which you probably don't have (and shouldn't). As mentioned in the comments, you edit code in Kubernetes by building a new container image and updating your Deployment (or similar) to use that image. It will then update all your pods.