如何为 AKS 中的出口流量保留 pod IP 地址？

Question

In Azure Kubernetes (AKS) if pod sending the traffic outside the cluster it get Natted to the Node IP address. For example, if pod nettools ( from node aks-agentpool-35359625-1 ) sending the traffic to Azure destinations that is outside of the cluster it get natted to 10.240.0.35. Is there a way to preserve the original source ( 10.240.0.48 )?

Cluster running with networking plugin azure and transparent networking mode

/AKS/cluster1 $ kubectl get nodes -o wide 
kubectl get pods -o wide
NAME                       STATUS   ROLES   AGE     VERSION    INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-agentpool-35359625-0   Ready    agent   6h13m   v1.15.10   10.240.0.66   <none>        Ubuntu 16.04.6 LTS   4.15.0-1071-azure   docker://3.0.10+azure
aks-agentpool-35359625-1   Ready    agent   6h13m   v1.15.10   10.240.0.35   <none>        Ubuntu 16.04.6 LTS   4.15.0-1071-azure   docker://3.0.10+azure
aks-agentpool-35359625-2   Ready    agent   6h13m   v1.15.10   10.240.0.4    <none>        Ubuntu 16.04.6 LTS   4.15.0-1071-azure   docker://3.0.10+azure
/AKS/cluster1 $ kubectl get pods -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP            NODE                       NOMINATED NODE   READINESS GATES
nettools   1/1     Running   0          21m   10.240.0.48   aks-agentpool-35359625-1   <none>           <none>

/AKS/cluster1 $ az aks show -g $rg --name $cluster --query "networkProfile"
{
  "dnsServiceIp": "10.0.0.10",
  "dockerBridgeCidr": "172.17.0.1/16",
  "loadBalancerProfile": null,
  "loadBalancerSku": "Basic",
  "networkMode": "transparent",
  "networkPlugin": "azure",
  "networkPolicy": null,
  "outboundType": "UDR",
  "podCidr": null,
  "serviceCidr": "10.0.0.0/16"
}

Answer 1

You actually can't do that, every time a Pod is destroyed and a new Pod is created, a new IP will be assigned to it. Pods are ephemeral and their IPs as well.

I believe that to achieve what you are looking for you need to use Services.

Services in Kubernetes are an "abstract way to expose an application running on a set of Pods as a network service." ( k8s documentation )

Again, you can access your pod by its IP and port that Kubernetes have given to it, but that's not a good practice as the Pods can die and another one will be created (if controlled by a Deployment/ReplicaSet) with a new IP, and everything on your app will start to fail.

If you expose your Deployment or Pod, the Service created will have an IP as well, and doesn't matter how many times your Pod is destroyed, because you will still be able to access it through the Service IP (o course when the new Pod is up and running, but I think you got the idea).

Answer 2

You can edit the azure-ip-masq-agent configmap to add the target private IP CIDR that you don't want it to be Masqueraded.

kubectl get pods --namespace kube-system -l "k8s-app=azure-ip-masq-agent" -o wide

kubectl describe configmap azure-ip-masq-agent-config --namespace kube-system

# Add x.x.x.x/x to the nonMasq CIDRs
kubectl edit configmap azure-ip-masq-agent-config --namespace kube-system