[英]TLS doesn't work with LoadBalancer backed Service in Kubernetes
I am trying to expose an application in my cluster by creating a service type as load balancer.我正在尝试通过将服务类型创建为负载均衡器来公开集群中的应用程序。 The reason for this is that I want this app to have a separate channel for communication.这样做的原因是我希望这个应用程序有一个单独的通信渠道。 I have a KOPS cluster.我有一个 KOPS 集群。 I want to use AWS's network load balancer so that it gets a static IP.我想使用 AWS 的网络负载均衡器,以便获得静态 IP。 When I create the Service with port 80 mapped to the port that the app is running on everything works but when I try to add port 443 it just times out.当我创建端口 80 映射到应用程序运行的端口的服务时,一切正常,但是当我尝试添加端口 443 时,它只是超时。
Here is the configuration that works -这是有效的配置 -
apiVersion: v1
metadata:
name: abc
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
labels:
app: abc
spec:
externalTrafficPolicy: Local
ports:
- name: http
port: 80
protocol: TCP
targetPort: 9050
selector:
app: abc
type: LoadBalancer
As soon as I add TLS support in the config file and deploy it.一旦我在配置文件中添加 TLS 支持并部署它。 The connection to the load balancer times out.与负载平衡器的连接超时。 How do I add TLS support to the load balancer?如何向负载均衡器添加 TLS 支持? I want to do it through the service and not through an ingress.我想通过服务而不是通过入口来做到这一点。 This is the configuration that doesn't work for me and when I paste the link in the browser, it times out.这是对我不起作用的配置,当我在浏览器中粘贴链接时,它超时了。
kind: Service
apiVersion: v1
metadata:
name: abc
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxxx
labels:
app: abc
spec:
externalTrafficPolicy: Local
ports:
- name: http
port: 443
protocol: TCP
targetPort: 9050
selector:
app: abc
type: LoadBalancer
You can use the tls & ssl termination您可以使用 tls & ssl 终止
apiVersion: v1
kind: Service
metadata:
name: test-service
annotations:
# Note that the backend talks over HTTP.
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
# TODO: Fill in with the ARN of your certificate.
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:{region}:{user id}:certificate/{id}
# Only run SSL on the port named "https" below.
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
spec:
selector:
app: test-pod
ports:
- name: http
port: 80
targetPort: 8080
- name: https
port: 443
targetPort: 8080
type: LoadBalancer
You can add the tls certficate in aws certificate manager and use the arn address of certificate to kubernetes service.您可以在 aws 证书管理器中添加 tls certficate 并使用证书的 arn 地址到 kubernetes 服务。
it's like in becked you can terminate the https connection and use the HTTP only.就像在 becked 中,您可以终止 https 连接并仅使用 HTTP。
you can also check this out : https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/你也可以看看这个: https : //aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/
https://github.com/kubernetes/kubernetes/issues/73297
EDIT :1编辑:1
service.beta.kubernetes.io/aws-load-balancer-type: nlb
if not work please try adding this annotation as per your loadbalancer type.如果不起作用,请尝试根据您的负载均衡器类型添加此注释。
You can now deploy ingress using NLB and SSL termination (https in NLB > http in service).您现在可以使用 NLB 和 SSL 终止(NLB 中的 https > 服务中的 http)部署入口。 Finally found a solution that worked for me, you can try to deploy the following ingress.yaml (make sure to update your cert ARN under deployment section):终于找到了对我有用的解决方案,您可以尝试部署以下 ingress.yaml(确保在部署部分更新您的证书 ARN):
---
# Source: nginx-ingress/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
name: nginx-ingress
---
# Source: nginx-ingress/templates/default-backend-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
name: nginx-ingress-backend
---
# Source: nginx-ingress/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
component: "controller"
heritage: Helm
release: nginx-ingress
name: nginx-ingress-controller
data:
server-snippet: |
listen 8000;
if ( $server_port = 80 ) {
return 308 https://$host$request_uri;
}
ssl-redirect: "false"
---
# Source: nginx-ingress/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
name: nginx-ingress
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- update
- watch
- apiGroups:
- extensions
- "networking.k8s.io" # k8s 1.14+
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- extensions
- "networking.k8s.io" # k8s 1.14+
resources:
- ingresses/status
verbs:
- update
---
# Source: nginx-ingress/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
name: nginx-ingress
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress
subjects:
- kind: ServiceAccount
name: nginx-ingress
namespace: ingress-nginx
---
# Source: nginx-ingress/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
name: nginx-ingress
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- update
- watch
- apiGroups:
- extensions
- "networking.k8s.io" # k8s 1.14+
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
- "networking.k8s.io" # k8s 1.14+
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
- ingress-controller-leader-nginx
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- create
- get
- update
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
# Source: nginx-ingress/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
name: nginx-ingress
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress
subjects:
- kind: ServiceAccount
name: nginx-ingress
namespace: ingress-nginx
---
# Source: nginx-ingress/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
namespace: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:##REPLACE WITH YOUR CERT ARN"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
component: "controller"
heritage: Helm
release: nginx-ingress
name: nginx-ingress-controller
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: special
selector:
app: nginx-ingress
release: nginx-ingress
app.kubernetes.io/component: controller
type: "LoadBalancer"
---
# Source: nginx-ingress/templates/default-backend-service.yaml
apiVersion: v1
kind: Service
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
component: "default-backend"
heritage: Helm
release: nginx-ingress
name: nginx-ingress-default-backend
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app: nginx-ingress
release: nginx-ingress
app.kubernetes.io/component: default-backend
type: "ClusterIP"
---
# Source: nginx-ingress/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
app.kubernetes.io/component: controller
name: nginx-ingress-controller
annotations:
{}
spec:
selector:
matchLabels:
app: nginx-ingress
release: nginx-ingress
replicas: 1
revisionHistoryLimit: 10
strategy:
{}
minReadySeconds: 0
template:
metadata:
labels:
app: nginx-ingress
release: nginx-ingress
component: "controller"
app.kubernetes.io/component: controller
spec:
dnsPolicy: ClusterFirst
containers:
- name: nginx-ingress-controller
image: "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0"
imagePullPolicy: "IfNotPresent"
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/nginx-ingress-default-backend
- --publish-service=$(POD_NAMESPACE)/nginx-ingress-controller
- --election-id=ingress-controller-leader
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/nginx-ingress-controller
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
runAsUser: 101
allowPrivilegeEscalation: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
- name: special
containerPort: 8000
protocol: TCP
readinessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
resources:
{}
hostNetwork: false
serviceAccountName: nginx-ingress
terminationGracePeriodSeconds: 60
---
# Source: nginx-ingress/templates/default-backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: ingress-nginx
labels:
app: nginx-ingress
chart: nginx-ingress-1.38.0
heritage: Helm
release: nginx-ingress
app.kubernetes.io/component: default-backend
name: nginx-ingress-default-backend
spec:
selector:
matchLabels:
app: nginx-ingress
release: nginx-ingress
replicas: 1
revisionHistoryLimit: 10
template:
metadata:
labels:
app: nginx-ingress
release: nginx-ingress
app.kubernetes.io/component: default-backend
spec:
containers:
- name: nginx-ingress-default-backend
image: "k8s.gcr.io/defaultbackend-amd64:1.5"
imagePullPolicy: "IfNotPresent"
args:
securityContext:
runAsUser: 65534
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 0
periodSeconds: 5
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 6
ports:
- name: http
containerPort: 8080
protocol: TCP
resources:
{}
serviceAccountName: nginx-ingress-backend
terminationGracePeriodSeconds: 60
Your annotation refers to https
port您的注释是指https
端口
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
but your port is named http
, change to https
但是您的端口名为http
,请更改为https
spec:
externalTrafficPolicy: Local
ports:
- name: https
port: 443
protocol: TCP
targetPort: 9050
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.