start a pod from inside a pod in kubernetes

Question

My goal is to have a kubernetes cluster running with stateless replicas of some kind of frontend which can schedule jobs.

Afterwards my plan is to have multiple runners (as pods) which are polling for scheduled jobs. Once they receive job data they should launch job executors (also as a pod) on demand.

This should look somehow like this:

  pod A                    pod B
 ________                __________
|        |      kube    |          |
| runner |    ------>   | executor |  
|________|              |__________|
                 .
                 .
  pod A'         .         pod B'
 ________                __________
|        |      kube    |          |
| runner |    ------>   | executor |  
|________|              |__________|

Basically I am kind of inspired by the gitlab architecture. https://docs.gitlab.com/runner/executors/kubernetes.html#kubernetes-executor-interaction-diagram

Therefore it would be really awesome if you could give me a hint where to start with. I was considering to use the python api of the cluster but in this I have to somehow make the cluster administration available to the runner.

So I'm really thankful for any idea or approach how to realize such an architecture of pod lifecycling.

Do you see any security risks with such an approach or is this reasonable securitywise?

Thank you very much for your help and hints

Answer 1

Here is a list of available kubernetes api clients . You will use a service account to make authorized calls with your api client against the kube-apiserver running in the controlplane node. I made a first idea draft for your architecture. Maybe that is inspiring to you. Good look with your project. Alternative to the database you may want to use a message queue.