我可以在多个 DevOps 产品团队之间安全地共享一个 k8s 集群吗？

Question

would there be a secure way to work with different product DevOps teams on the same k8s cluster? How can I isolate workloads between the teams? I know there is k8s rbac and namespaces available, but is that secure to run different prod workloads? I know istio but as I understood there is no direct answer to my Südasien. How can we handle different in ingress configuration from different teams in the same cluster? If not securely possible to isolate workloads how do you orchestrate k8s clusters to reduce maintenance.

Thanks a lot!

Answer 1

The answer is: it depends. First, Kubernetes is not insecure by default and containers give a base layer of abstraction. The better questions are:

• How many isolation do you need?
• Whats about user management?
• Do you need to encrypt traffic between your workload?

Isolation Levels

If you need strong isolation between your workloads (and i mean really strong), do yourself a favor and use different clusters. There may be some business cases where you need guarantee that some kind of workload is not allowed to run on the same (virtual) machine. You could also try to do this by adding nodes that are only for one of your sub-projects and use Affinities and Anti-Affinities to handle the scheduling. But if need this level of isolation, you'll probably ran into problems when thinking about log aggregation, metrics or in general any point where you have a component that's used across all of your services.

For any other use case: Build one cluster and divide by namespaces. You could even create a couple ingress-controllers which belong just to one of your teams.

User Management

Managing RBAC and users by hand could be a little bit tricky. Kubernetes itself supports OIDC-Tokens . If you already use OIDC for SSO or similar, you could re-use your tokens to authenticate users in Kubernetes. I've never used this, so i can't tell about role mapping using OIDC.

Another solution would be Rancher or another cluster orchestrating tool. I can't tell about the other, but Rancher comes with built-in user management. You could also create projects to group several namespaces for one of your audiences.

Traffic Encryption

By using a service mesh like Istio or Linkerd you can encrypt traffic between your pods. Even if it sounds seductive to encrypt your workload, make clear if you really need this. Service meshes come with some downsides, eg resource usage. Also you have one more component that needs to be managed and updated.