Kubernetes Job: exactly one pod

Question

I want to run only one pod of my kubernetes app at a time(relaunch in case of failure), I am using job controller .

But as per documentations, kubernetes may launch more than one pods and will eventually achieve specified replicas. Is there any way to achieve exactly one pod at a time or any recommended design pattern for such use cases.

My app is reading data from HDFS and writing it to a message queue. It exits after processing all the files. I want to minimize possibility of writing duplicate records.

Answer 1

I suggest you use replicasets for this. Set the number of replica to 1. More here https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#when-to-use-a-replicaset

Answer 2

In principle, in the case of Jobs with no paralellism implied, there shouldn't be this kind of "race condition" ("should be 1" according to the documentation [1]). The job would be rescheduled only if an attempt fails. Did you come across the situation where 2 pods from the same job were being executed at the same time?

In any case, if you want to be completely sure, you may want to implement an extra coordination method or external solution.

---

[1] https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/

Answer 3

If I understand your question correctly I think you are looking for: .spec.strategy.rollingUpdate.maxSurge

If you set this to 0 then the existing pods will be killed before starting an new one.