[英]SharedIndexInformer (Fabric8 kubernetes-client) watches only pods of its own namespace when run in the cluster
I am trying to build a Kubernetes controller using the excellent Fabric8 Kubernetes client for Java ( https://github.com/fabric8io/kubernetes-client ).我正在尝试使用用于 Java 的优秀 Fabric8 Kubernetes 客户端( https://github.com/fabric8io/kubernetes-client )构建一个 Kubernetes 控制器。 As of now I use the version
4.10.3
.截至目前,我使用版本
4.10.3
。
For that purpose I am constructing a SharedIndexInformer
to properly watch resources events emitted by the cluster.为此,我正在构建一个
SharedIndexInformer
来正确监视集群发出的资源事件。 I will take pods as resources example here.我将在这里以 Pod 为资源示例。
So the SharedIndexInformer
is constructed following this piece of code:所以
SharedIndexInformer
是按照这段代码构建的:
SharedIndexInformer<Pod> sharedIndexInformer = kubernetesClient.informers().sharedIndexInformerFor(
objectClass,
objectClassList,
10 * 60 * 1000);
Following, lot of code to attach events handler, start the indexer, have a reconciliation loop and so on.接下来,很多代码来附加事件处理程序,启动索引器,有一个协调循环等等。
The indexer is working perfectly fine when started from my local machine, and I see all pods being listed.当从我的本地机器启动时,索引器工作得很好,我看到所有的 pod 都被列出来了。 However, when I run it on a pod in my cluster (with RBAC properly defined), I see only the pods for the namespace where the pod is run on.
但是,当我在集群中的 pod 上运行它时(正确定义了 RBAC),我只能看到运行 pod 的命名空间的 pod。
I checked explicitly in the pod that, using kubectl
, the associated service account was capable to list all pods in the cluster, and not only in the current namespace.我在 pod 中明确检查,使用
kubectl
,关联的服务帐户能够列出集群中的所有 pod,而不仅仅是在当前命名空间中。
What am I missing?我错过了什么?
Thanks in advance for your help!在此先感谢您的帮助!
I think this is due to the difference between how KubernetesClient
creates it's Config
when outside Kubernetes Cluster or inside a Pod
.我认为这是由于
KubernetesClient
在 Kubernetes Cluster 外部或Pod
内部创建它的Config
的方式不同。 In the former case, KubernetesClient
usually reads from your ~/.kube/config
and connection information like token and namespace are picked up from your current context in your ~/.kube/config
file.在前一种情况下,
KubernetesClient
通常从您的~/.kube/config
读取,并且连接信息(例如令牌和命名空间)是从您的~/.kube/config
文件中的当前上下文中~/.kube/config
。
However, when KubernetesClient
is inside a Pod;但是,当
KubernetesClient
在 Pod 内时; it picks up connection Config
information from loaded ServiceAccount
, See Config.java .它从加载的
ServiceAccount
获取连接Config
信息,请参阅Config.java 。 Bearer token gets picked from /var/run/secrets/kubernetes.io/serviceaccount/token
and the default namespace to be used for namespaced API operations is picked from /var/run/secrets/kubernetes.io/serviceaccount/namespace
.不记名令牌从
/var/run/secrets/kubernetes.io/serviceaccount/token
选取,用于命名空间 API 操作的默认命名空间从/var/run/secrets/kubernetes.io/serviceaccount/namespace
选取。 You can find more about it in Kubernetes Docs: Accessing API from a Pod .您可以在Kubernetes Docs: Accessing API from a Pod 中找到更多相关信息。 I think
KubernetesClient
is picking this namespace while loading the Config
.我认为
KubernetesClient
在加载Config
时正在选择这个命名空间。
I think KubernetesClient
is not handling this case properly.我认为
KubernetesClient
没有正确处理这种情况。 This should be fixed there.这应该在那里修复。 There is already an issue filed there: https://github.com/fabric8io/kubernetes-client/issues/2514
那里已经有一个问题: https : //github.com/fabric8io/kubernetes-client/issues/2514
I'm not sure if right now informers can detect whether they are in-cluster or outside(This is only known till we load Config
).我不确定现在的告密者是否可以检测到他们是在集群内还是在集群外(这只有在我们加载
Config
时才知道)。 Right now, informers provide way to specify namespace using OperationContext
:现在,informers 提供了使用
OperationContext
指定命名空间的方法:
SharedInformerFactory sharedInformerFactory = client.informers();
SharedIndexInformer<Pod> podInformer = sharedInformerFactory.sharedIndexInformerFor(
Pod.class,
PodList.class,
new OperationContext().withNamespace("default"),
30 * 1000L);
Maybe for overriding this namespace being loaded from ServiceAccount
we can allow setting null
namespace:也许为了覆盖从
ServiceAccount
加载的这个命名空间,我们可以允许设置null
命名空间:
SharedIndexInformer<Pod> podInformer = sharedInformerFactory.sharedIndexInformerFor(
Pod.class,
PodList.class,
new OperationContext().withNamespace(null), // -> Doesn't work; Ideally should Watch in all namespaces,
30 * 1000L);
Update:更新:
The underlying issue seems to be fixed in v4.13.0 .潜在问题似乎已在v4.13.0中解决。 I've tested this on this demo project: https://github.com/r0haaaan/fabric8-kubernetes-java-informer-in-pod .
我已经在这个演示项目中对此进行了测试: https : //github.com/r0haaaan/fabric8-kubernetes-java-informer-in-pod 。 It runs SharedIndexInformers in a project and deploy to Kubernetes using Kubernetes Maven Plugin .
它在项目中运行 SharedIndexInformers 并使用Kubernetes Maven Plugin部署到 Kubernetes。 When I check logs, I can see that all pods seem to be listed:
当我检查日志时,我可以看到似乎列出了所有 pod:
fabric8-kubernetes-java-informers-in-pod : $ mvn k8s:log
[INFO] Scanning for projects...
[INFO]
[INFO] --------< org.example:fabric8-kubernetes-java-informers-in-pod >--------
[INFO] Building fabric8-kubernetes-java-informers-in-pod 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- kubernetes-maven-plugin:1.0.2:log (default-cli) @ fabric8-kubernetes-java-informers-in-pod ---
[INFO] k8s: Using Kubernetes at https://192.168.39.24:8443/ in namespace default with manifest /home/rohaan/work/repos/fabric8-kubernetes-java-informers-in-pod/target/classes/META-INF/jkube/kubernetes.yml
[INFO] k8s: Using namespace: default
[INFO] k8s: Watching pods with selector LabelSelector(matchExpressions=[], matchLabels={app=fabric8-kubernetes-java-informers-in-pod, provider=jkube, group=org.example}, additionalProperties={}) waiting for a running pod...
[INFO] k8s: [NEW] fabric8-kubernetes-java-informers-in-pod-6f957b6b59-tpbgd status: Running Ready
[INFO] k8s: [NEW] Tailing log of pod: fabric8-kubernetes-java-informers-in-pod-6f957b6b59-tpbgd
[INFO] k8s: [NEW] Press Ctrl-C to stop tailing the log
[INFO] k8s: [NEW]
[INFO] k8s: Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
[INFO] k8s: INFO exec java -javaagent:/usr/share/java/jolokia-jvm-agent/jolokia-jvm.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -javaagent:/usr/share/java/prometheus-jmx-exporter/jmx_prometheus_javaagent.jar=9779:/opt/jboss/container/prometheus/etc/jmx-exporter-config.yaml -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=100m -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/fabric8-kubernetes-java-informers-in-pod-1.0-SNAPSHOT-jar-with-dependencies.jar
[INFO] k8s: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[INFO] k8s: SLF4J: Defaulting to no-operation (NOP) logger implementation
[INFO] k8s: SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] k8s: WARNING: An illegal reflective access operation has occurred
[INFO] k8s: WARNING: Illegal reflective access by org.jolokia.util.ClassUtil (file:/usr/share/java/jolokia-jvm-agent/jolokia-jvm.jar) to constructor sun.security.x509.X500Name(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String)
[INFO] k8s: WARNING: Please consider reporting this to the maintainers of org.jolokia.util.ClassUtil
[INFO] k8s: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
[INFO] k8s: WARNING: All illegal access operations will be denied in a future release
[INFO] k8s: Nov 10, 2020 5:37:50 PM io.fabric8.testing.SimpleSharedInformerRun main
[INFO] k8s: INFO: k8s.getConfiguration().getNamespace(): default
[INFO] k8s: I> No access restrictor found, access to any MBean is allowed
[INFO] k8s: Jolokia: Agent started with URL https://172.17.0.6:8778/jolokia/
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: default/fabric8-kubernetes-java-informers-in-pod-6f957b6b59-tpbgd
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: istio-system/istio-ingressgateway-64cfb9d44b-kk5ft
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: istio-system/istiod-7684b696d6-fhzwt
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/coredns-f9fd979d6-g4htj
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/etcd-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/kube-apiserver-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/kube-controller-manager-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/kube-proxy-tpsrg
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/kube-scheduler-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/metrics-server-d9b576748-4w6jz
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: kube-system/storage-provisioner
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onAdd
[INFO] k8s: INFO: ADDED: rokumar/multi-container-pod
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: default/fabric8-kubernetes-java-informers-in-pod-6f957b6b59-tpbgd
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: istio-system/istio-ingressgateway-64cfb9d44b-kk5ft
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: istio-system/istiod-7684b696d6-fhzwt
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/coredns-f9fd979d6-g4htj
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/etcd-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/kube-apiserver-minikube
[INFO] k8s: [NEW] fabric8-kubernetes-java-informers-in-pod-6f957b6b59-tpbgd status: Running Ready
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/kube-controller-manager-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/kube-proxy-tpsrg
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/kube-scheduler-minikube
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/metrics-server-d9b576748-4w6jz
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: kube-system/storage-provisioner
[INFO] k8s: Nov 10, 2020 5:37:52 PM io.fabric8.testing.SimpleSharedInformerRun$1 onUpdate
[INFO] k8s: INFO: UPDATED: rokumar/multi-container-pod
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.