Kubernetes 对具有 IP 白名单的外部端点的出站调用

Question

We are using Kubernetes on google cloud's Google Kubernetes Engine. Our system dynamically generates instances based on request and these instances call an external web service. The external service generates images and the bandwidth usage per instance is not small.

This external web service has an IP whitelisting configured.

Is there any way that I can funnel all the requests going from the selected pods (they are grouped within a node pool) to the external service with a single IP?

Answer 1

The answer is Yes , there are actually several ways one can achieve this. I will answer a simple way to get this done. By tunnelling through a proxy server.

It could also be done assigning external ips to all your nodes and allowing them from webservice, but many engineers don't prefer doing it because no one wants to expose the nodes to the external world for a million security reasons.

Add a separate very small may be nano VM within the same cluster and install a HAProxy or Nginx or your favourite proxy. Or install the proxy on one of the instances you already have but make sure it has external ip attached to it, and it should be inside your cluster in order to reduce any latency issues.

Now bind the url in the proxy to accept connection to a particular port and route them to your instance that has your external webservice. This is an example of HAProxy code how it would look like.

listen port_2020
  bind :2020
  mode tcp
  server external-web-service externalwebservice.mycompany.com:443 check

After the completion of this setup. Let's assume your k8s is running masters at 10.0.1.0/24 and nodes at 10.0.2.0/24 . And added this addition proxy service somewhere at 10.10.1.101/32 with an external ip of 52.*.*.* with in the same VPC. Now all you have to do is open communication on 10.10.1.101 to accept communications to port 2020 from 10.0.2.0/24 .

Now your pods have to keep polling 10.10.1.101:2020/api/health/check instead of external webservice directly.

And now you can waitlist just the proxy vm ip 52.*.*.* on your webservice vm without any issues.

This is just an example of how it could be done. But there are several approaches to get this done. There are many advanced ways of doing this using a sidecar as well.

Hope this is helpful.