Kubernetes使用api观看Pod活动

Question

We are interested in running certain commands as pods and services, as they start or stop. Using the life-cycle hooks in the yml files does not work for us, since these commands are not optional. We have considered running a watcher pod that uses the watch api to run these commands. But we can't figure out how to use the watch api so that it does not keep sending the same events again and again. Is there a way to tell the watch api to only send new events since connection was opened? If expecting a stateful watch api is unreasonable, will it be possible to pass it a timestamp or a monotonically increasing id to avoid getting already seen events?

Basically what we are doing now we are running a pod with a daemon process that communicates with the api. we can find the events as stream. But we are interested to run some task when a pod created or deleted.

Answer 1

I have found the answer. In case anyone else is watching.

There is a much better system to watch resource and handle events with custom tasks with pkg/controller/framework package

I found the step like this,

1. initiate a framework.NewInFormer
2. Run the controller
3. the NewInFormer loads with your custom event handlers that will call when the events occured.

Answer 2

I'd recommend using the client from the kube repo. Why are the lifecycle hooks not working for your use case?

Answer 3

Run kube proxy to use curl without authentication

kubectl proxy 

List all events with a watch;

curl -s 127.0.0.1:8001/api/v1/watch/events 

Run the curl to watch the events and filter it with jq for pod starts and stops.

curl -s 127.0.0.1:8001/api/v1/watch/events | jq --raw-output \ 'if .object.reason == "Started" then . elif .object.reason == "Killing" then . else empty end | [.object.firstTimestamp, .object.reason, .object.metadata.namespace, .object.metadata.name] | @csv'

More details

Answer 4

In case it is in-cluster you may do it like this in golang:

package main

import (
    "fmt"
    "time"

    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/pkg/api/v1"
    "k8s.io/client-go/tools/cache"
    "k8s.io/client-go/pkg/fields"
    "k8s.io/client-go/rest"
)

func main() {
    config, err := rest.InClusterConfig()
    if err != nil {
        panic(err.Error())
    }

    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        panic(err.Error())
    }

    watchlist := cache.NewListWatchFromClient(clientset.Core().RESTClient(), "pods", v1.NamespaceDefault, 
       fields.Everything())
    _, controller := cache.NewInformer(
        watchlist,
        &v1.Pod{},
        time.Second * 0,
        cache.ResourceEventHandlerFuncs{
            AddFunc: func(obj interface{}) {
                fmt.Printf("add: %s \n", obj)
            },
            DeleteFunc: func(obj interface{}) {
                fmt.Printf("delete: %s \n", obj)
            },
            UpdateFunc:func(oldObj, newObj interface{}) {
                fmt.Printf("old: %s, new: %s \n", oldObj, newObj)
            },
        },
    )
    stop := make(chan struct{})
    go controller.Run(stop)
}

Answer 5

You say the lifecycle hooks commands are not optional, but they are indeed optional.

Hook handler implementations
Containers can access a hook by implementing and registering a handler for that hook. There are two types of hook handlers that can be implemented for Containers:
Exec - Executes a specific command, such as pre-stop.sh, inside the cgroups and namespaces of the Container. Resources consumed by the command are counted against the Container.
HTTP - Executes an HTTP request against a specific endpoint on the Container.

From here: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/