将 docker-compose nginx.conf 迁移到 kubernetes nginx ingress

Question

I am trying to convert my docker-compose nginx.conf to kubernetes nginx ingress controller. 这是我与 docker-compose 一起使用的 nginx.conf。

server {
    listen 3000;
    root /srv/www/static;
    sendfile on;
    gzip on;
    gzip_comp_level 1;
    gzip_proxied any;
    gzip_vary on;
    gzip_types application/json text/plain;

    location /api/main {
        rewrite /api/main(.*)$1 break;
        proxy_pass http://main:4000;
        proxy_set_header content-type "application/json";
    }
    location /api/login {
        rewrite /api/login(.*)$1 break;
        proxy_pass http://login:4000;
        proxy_set_header content-type "application/json";
    }
}

这是 nginx 入口 controller。 我启用了入口插件。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - host: test.com
    http:
      paths:
      - path: /api/login(.*)
        pathType: Prefix
        backend:
          service:
            name: login-app
            port:
              number: 4000
      - path: /api/main(.*)
        pathType: Prefix
        backend:
          service:
            name: main-app
            port:
              number: 4000

kubectl 获取入口：

NAME             CLASS    HOSTS        ADDRESS   PORTS   AGE
test-ingress   <none>   test.com                 80      20h

kubectl 描述入口测试入口：

Name:             test-ingress
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  test.com  
              /api/login(.*)   login-app:4000 (10.1.146.185:4000)
              /api/main(.*)    main-app:4000 (10.1.146.186:4000)
Annotations:  nginx.ingress.kubernetes.io/rewrite-target: /$1
Events:       <none>

If I go to test.com:80 I get an nginx 404 error so I believe its up and running but any request that I make to an endpoint also returns 404. I also defined test.com in my local machine etc/hostname and I我也在使用 microk8s --classic 而不是 minikube。

我设置它的方式是我需要能够向端点发送请求并将 url 路径转发到相应的服务。 所以 test.com/api/main/orders/orderStatus 将被重写为 main-service/api/main/orders/orderStatus 但这会返回 404 错误我认为这可能是我为 kubernetes 设置重写注释的方式，因为我的 nginx.conf 用于 docker-compose 工作正常。

kubectl -n 入口日志 nginx-ingress-microk8s-controller-5h5qm：

NGINX Ingress controller
  Release:       v0.35.0
  Build:         54ad65e32bcab32791ab18531a838d1c0f0811ef
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.2

-------------------------------------------------------------------------------

I0210 20:46:20.300962       6 flags.go:205] Watching for Ingress class: public
W0210 20:46:20.301239       6 flags.go:208] Only Ingresses with class "public" will be processed by this Ingress controller
W0210 20:46:20.302300       6 flags.go:252] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0210 20:46:20.302548       6 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0210 20:46:20.431594       6 main.go:231] Creating API client for https://10.152.183.1:443
I0210 20:46:20.492885       6 main.go:275] Running in Kubernetes cluster version v1.20+ (v1.20.1-34+e7db93d188d0d1) - git (clean) commit e7db93d188d0d12f2fe5336d1b85cdb94cb909d3 - platform linux/amd64
I0210 20:46:20.917856       6 main.go:105] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
I0210 20:46:21.002278       6 main.go:113] Enabling new Ingress features available since Kubernetes v1.18
I0210 20:46:22.130699       6 nginx.go:263] Starting NGINX Ingress controller
I0210 20:46:22.315064       6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-ingress-udp-microk8s-conf", UID:"73215902-b0b2-4da2-b402-5aa68d4a5be6", APIVersion:"v1", ResourceVersion:"668", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-ingress-udp-microk8s-conf
I0210 20:46:22.315231       6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-load-balancer-microk8s-conf", UID:"8f8cd494-7d19-4168-8f05-e4992ea3f4de", APIVersion:"v1", ResourceVersion:"666", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-load-balancer-microk8s-conf
I0210 20:46:22.401763       6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-ingress-tcp-microk8s-conf", UID:"885189d0-e5ca-47e2-98f4-8982d813606a", APIVersion:"v1", ResourceVersion:"667", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-ingress-tcp-microk8s-conf
I0210 20:46:23.347072       6 store.go:363] ignoring add for ingress test-ingress based on annotation kubernetes.io/ingress.class with value 
I0210 20:46:23.431950       6 nginx.go:307] Starting NGINX process
I0210 20:46:23.432052       6 leaderelection.go:242] attempting to acquire leader lease  ingress/ingress-controller-leader-public...
I0210 20:46:23.444902       6 controller.go:141] Configuration changes detected, backend reload required.
I0210 20:46:25.899037       6 leaderelection.go:252] successfully acquired lease ingress/ingress-controller-leader-public
I0210 20:46:25.899076       6 status.go:86] new leader elected: nginx-ingress-microk8s-controller-5h5qm
I0210 20:46:26.061674       6 controller.go:157] Backend successfully reloaded.
I0210 20:46:26.061807       6 controller.go:166] Initial sync, sleeping for 1 second.
W0210 20:46:27.063786       6 controller.go:184] Dynamic reconfiguration failed: Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
E0210 20:46:27.063892       6 controller.go:188] Unexpected failure reconfiguring NGINX:
Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
W0210 20:46:27.063941       6 queue.go:130] requeuing initial-sync, err Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
I0210 20:46:27.064158       6 controller.go:141] Configuration changes detected, backend reload required.
I0210 20:46:27.398413       6 controller.go:157] Backend successfully reloaded.
I0210 20:46:27.398500       6 controller.go:166] Initial sync, sleeping for 1 second.

部署示例 mainDep.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: main-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: main-app
  template:
    metadata:
      labels:
        app: main-app
    spec:
      containers:
        - name: main-app
          image: peking-main:latest
          imagePullPolicy: Never
          ports:
          - containerPort: 4000
          resources:
            limits:
              memory: "100Mi"
              cpu: "100m"

服务示例 mainService.yaml

apiVersion: v1
kind: Service
metadata:
  name: main-app
  labels:
    app: main-app
spec:
  type: LoadBalancer
  selector:
    app: main-app
  ports:
    - port: 80
      targetPort: 4000

kubectl 获取 svc

login-app    LoadBalancer   10.152.183.107   <pending>     80:30412/TCP   16m
main-app     LoadBalancer   10.152.183.108   <pending>     80:31369/TCP   16m

kubectl 描述 svc 主应用程序

Name:                     main-app
Namespace:                default
Labels:                   app=main-app
Annotations:              <none>
Selector:                 app=main-app
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.152.183.108
IPs:                      10.152.183.108
Port:                     <unset>  80/TCP
TargetPort:               4000/TCP
NodePort:                 <unset>  31369/TCP
Endpoints:                10.1.146.171:4000
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Answer 1

所以我找到了一个解决方法。 我将我能找到的每个端口（例如应用程序和 dockerfile）更改为 80，并将入口的 api 版本更改为 v1beta。

''' apiVersion：networking.k8s.io/v1beta1 种类：入口元数据：名称：nginx 规范：规则：

• 主机：test.com http：路径：
  • 路径：/api/login 后端：serviceName：login-app servicePort：80
  • path: /api/main backend: serviceName: main-app servicePort: 80 ''' 虽然它有效，但每当我应用它时都会发出警告。