如何在 Azure AKS 上的一个入口控制器上创建多个负载均衡器 IP 地址

Question

I'm trying to setup multiple services on one k8s cluster, with one Ingress controller in front that does tls termination for all services. This is a good example: https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/examples/multi-tls/multi-tls.yaml

I initially followed this example: https://github.com/brunoterkaly/ingress , and then expanded it to have multiple tls services.

By exposing my nginx replication controller AKS on Azure automatically creates a loadbalancer and a public ip address, to which I can apply an A record: kubectl expose rc nginx-ingress-rc --port="80,443" --type="LoadBalancer"

However, I also want a second A record, that points to the same ip address (I guess?), so that I can access my ingress controller from different domains. I can't figure out how to let AKS create a second one for that purpose?

Answer 1

Maybe this is a bit too late and not exactly what the original post was asking for. Unofficially you can create multiple services for your ingress controller and thus also map multiple IPs. It is only limitation of single k8s service of type LoadBalancer that it can refer only single IP address.

In my case, I have AKS cluster with many namespaces and different applications reachable via different URLs. Each URL has different public IP address because of historical reasons:

Example:

first.example.com -> 1.2.3.4
second.example.com -> 5.6.7.8
...we could go on, IPs are just made up!

I wanted to install nginx ingress into AKS that would handle routing for all namespaces instead of handling each application with dedicated LoadBalancer and reverse proxy pair. For that I followed steps described here and provided just one of the IPs as controller.service.loadBalancerIP=”1.2.3.4” during ingress deployment process (using helm).

After this steps, I could see these services:

PS C:\> kubectl get service -n ingress-nginx
NAME                                   TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
ingress-nginx-controller               LoadBalancer   10.0.141.84    1.2.3.4         80:30065/TCP,443:31444/TCP   14h
ingress-nginx-controller-admission     ClusterIP      10.0.168.127   <none>          443/TCP                      14h

After that I manually created another service ingress-nginx-controller-second with exactly same selector labels and next loadBalancerIP: 5.6.7.8 .

After this steps, I could see these services:

PS C:\> kubectl get service -n ingress-nginx
NAME                                   TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
ingress-nginx-controller               LoadBalancer   10.0.141.84    1.2.3.4         80:30065/TCP,443:31444/TCP   14h
ingress-nginx-controller-admission     ClusterIP      10.0.168.127   <none>          443/TCP                      14h
ingress-nginx-controller-second        LoadBalancer   10.0.204.118   5.6.7.8         80:31275/TCP,443:32751/TCP   10m

Now if I list example ingresses that I defined for two applications, you can see, that both have the same first public IP shown. This is because that IP is used by ingress as a publish IP. Nevertheless, routing works nicely, since second service forwards all traffic to the same ingress-nginx-controller and you are able to do routing based on different hosts/paths just like always.

PS C:\Users\sk1u04h9> kubectl get ingress -A
NAMESPACE   NAME             CLASS    HOSTS                ADDRESS   PORTS     AGE
first       first-ingress    <none>   first.example.com    1.2.3.4   80, 443   14h
second      second-ingress   <none>   second.example.com   1.2.3.4   80, 443   5m

I hope this helps to someone trying to migrate to nginx ingress without the need to have upfront all URLs pointing to single public IP. You can request to have all URLs pointing to just one IP if needed after this migration step and then you can of course remove all ingress-nginx-controller-* services, that will not be needed anymore.

Answer 2

Meanwhile I understand Ingress a bit better. Only one ip address is created for the ingress controller, which can support path and host based routing. See https://learn.microsoft.com/en-us/azure/aks/ingress for an example. I'll just need to configure my DNS with different CNAME records for the A record that is associated with the Azure public IP address. If I want to use multiple *.cloudapp.azure.com fqdn's for my services, what I was trying to achieve at first, I'll have to use Azure DNS.