有 1 个传出 IP 用于 kube.netes 出口流量

Question

Current set-up

Cluster specs: Managed Kube.netes on Digital Ocean

Goal

My pods are accessing some websites but I want to use a proxy first.

Problem

The proxy I need to use is only taking 1 IP address in an "allow-list".

My cluster is using different nodes, with node-autoscaler so I have multiple and changing IP addresses.

Solutions I am thinking about

• Setting-up a proxy (squid? nginx?) outside of the cluster (Currently not working when I access an HTTPS website)
• Istio could let me set-up a gateway? (No knowledge of Istio)
• Use GCP managed K8s, and follow the answers on Kube.netes cluster outgoing traffic IP . But all our stack is on Digital Ocean and the pricing is better there.

---

I am curious to know what is the best practice, easiest solution or if anyone experienced such use-case before:)

Best

Answer 1

You could set up all your traffic to go through istio-egressgateway .

Then you could manipulate the istio-egressgateway to always be deployed on the same node of the cluster, and whitelist that IP address.

Pros: super easy. BUT. If you are not using Istio already, to set up Istio just for this is may be killing a mosquito with a bazooka.

Cons: Need to make sure the node doesn't change the IP address. Otherwise the istio-egressgateway itself might not get deployed (if you do not have the labels added to the new node), and you will need to reconfigure everything for the new node (new IP address). Another con might be the fact that if the traffic goes up, there is an HPA, which will deploy more replicas of the gateway, and all of them will be deployed on the same node. So, if you are going to have lots of traffic, may be it would be a good idea to isolate one node, just for this purpose.

Another option would be as you are suggesting; a proxy. I would recommend an Envoy proxy directly. I mean, Istio is going to be using Envoy anyways right? So, just get the proxy directly, put it in a pod, do the same thing as I mentioned before; node affinity, so it will always run on the same node, so it will go out with the same IP.

Pros: You are not installing entire service mesh control plane for one tiny thing.

Cons: Same as before, as you still have the issue of the node IP change if something goes wrong, plus you will need to manage your own Deployment object, HPA, configure the Envoy proxy, etc. instead of using Istio objects (like Gateway and a VirtualService ).

Finally, I see a third option; to set up a NAT gateway outside the cluster, and configure your traffic to go through it.

Pros: You won't have to configure any kube.netes object, therefor there will be no need to set up any node affinity, therefor there will be no node overwhelming or IP change. Plus you can remove the external IP addresses from your cluster, so it will be more secure (unless you have other workloads that need to reach inte.net directly). Also, probably having a single node configured as NAT will be more resilient then a kube.netes pod, running in a node.

Cons: May be a little bit more complicate to set up?

And there is this general Con, that you can whitelist only 1 IP address, so you will always have a single point of failure. Even NAT gateway; it still can fail.

The GCP static IP won't help you. What is suggesting the other post is to reserve an IP address, so you can re-use it always. But it's not that you will have that IP address automatically added to a random node that goes down. Human intervention is needed. I don't think you can have one specific node to have a static IP address, and if it goes down, the new created node will pick the same IP. That service, to my knowledge, doesn't exist.

Now, GCP does offer a very resilient NAT gateway. It is managed by Google, so shouldn't fail. Not cheap though.