将 blazor 服务器应用程序部署到 Kubernetes 时出现问题

Question

Lately I am trying to do a test deploy of a Blazor server app on locally hosted Kubernetes instance running on docker desktop.

I managed to correctly spin up the app in a container, migrations were applied etc, logs are telling me that the app is running and waiting:

[17:54:39 INF] Starting up...

[17:54:39 WRN] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.

[17:54:40 WRN] No XML encryptor configured. Key {9d879e73-4b80-4e63-8761-7ad2922164a0} may be persisted to storage in unencrypted form.

[17:54:40 INF] AuthorizationPolicy Configuration started...

[17:54:40 INF] Policy 'LocationMustBeSady' was configured successfully.

[17:54:40 INF] AuthorizationPolicy Configuration completed.

[17:54:41 INF] Now listening on: http://[::]:80

[17:54:41 INF] Application started. Press Ctrl+C to shut down.

[17:54:41 INF] Hosting environment: Production

[17:54:41 INF] Content root path: /app

The problem is when I am trying to connect from the local machine to said application using address set up in hosts file ( 127.0.0.1 scp.com )

My current settings are as follows:

Application Dockerfile (builds fine):

# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /source
EXPOSE 5000
EXPOSE 5001
EXPOSE 5005
EXPOSE 80
EXPOSE 443
ENV ASPNETCORE_ENVIRONMENT="docker"

LABEL name="ScanApp"

# copy csproj and restore as distinct layers
COPY ScanApp/*.csproj ScanApp/
COPY ScanApp.Application/*.csproj ScanApp.Application/
COPY ScanApp.Common/*.csproj ScanApp.Common/
COPY ScanApp.Domain/*.csproj ScanApp.Domain/
COPY ScanApp.Infrastructure/*.csproj ScanApp.Infrastructure/
COPY ScanApp.Tests/*.csproj ScanApp.Tests/
RUN dotnet restore ScanApp/ScanApp.csproj

# copy and build app and libraries
COPY ScanApp/ ScanApp/
COPY ScanApp.Application/ ScanApp.Application/
COPY ScanApp.Common/ ScanApp.Common/
COPY ScanApp.Domain/ ScanApp.Domain/
COPY ScanApp.Infrastructure/ ScanApp.Infrastructure/
COPY ScanApp.Tests/ ScanApp.Tests/
WORKDIR /source/ScanApp
RUN dotnet build -c release --no-restore

# test stage -- exposes optional entrypoint
# target entrypoint with: docker build --target test
FROM build AS test
WORKDIR /source/ScanApp.Tests
COPY tests/ .
ENTRYPOINT ["dotnet", "test", "--logger:trx"]

FROM build AS publish
RUN dotnet publish -c release --no-build -o /app

# final stage/image
# FROM mcr.microsoft.com/dotnet/runtime:5.0
FROM mcr.microsoft.com/dotnet/aspnet:5.0
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "ScanApp.dll"]

Kubernetes deployment for said app plus cluster ip - scanapp-depl.yaml :

apiVersion: apps/v1
kind: Deployment
metadata:
    name: scanapp-depl
spec:
    replicas: 1
    selector:
        matchLabels:
            app: scanapp
    template:
        metadata:
            labels:
                app: scanapp
        spec:
            containers:
                - name: scanapp
                  image: scanapp:1.0
---
apiVersion: v1
kind: Service
metadata:
    name: scanapp-clusterip-srv
spec:
    type: ClusterIP
    selector:
        app: scanapp
    ports:
        - name: ui
          protocol: TCP
          port: 8080
          targetPort: 80
        - name: ui2
          protocol: TCP
          port: 8081
          targetPort: 443
        - name: scanapp0
          protocol: TCP
          port: 5000
          targetPort: 5000
        - name: scanapp1
          protocol: TCP
          port: 5001
          targetPort: 5001
        - name: scanapp5
          protocol: TCP
          port: 5005
          targetPort: 5005

NodePort config - scanapp-np-srv.yaml

apiVersion: v1
kind: Service
metadata:
    name: scanappnpservice-srv
spec:
    type: NodePort
    selector:
        app: scanapp
    ports:    
        - name: ui
          port: 8080
          targetPort: 80
        - name: scanapp0
          protocol: TCP
          port: 5000
          targetPort: 5000
        - name: scanapp1
          protocol: TCP
          port: 5001
          targetPort: 5001
        - name: scanapp5
          protocol: TCP
          port: 5005
          targetPort: 5005

Ingress config - ingress-srv.yaml :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-srv
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /    
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "affinity"
    nginx.ingress.kubernetes.io/session-cookie-expires: "14400"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "14400"
spec:
  rules:
      - host: scp.com
        http:
            paths:
            - path: /
              pathType: Prefix
              backend:
                service:
                  name: scanapp-clusterip-srv
                  port:
                    number: 8080

In addition, there is also working sql server config with cluster ip and load balancer:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: mssql-depl
spec:
    replicas: 1
    selector:
        matchLabels:
            app: mssql
    template:
        metadata:
            labels:
                app: mssql
        spec:
            containers:
                - name: mssql
                  image: mcr.microsoft.com/mssql/server:2019-latest
                  ports:
                      - containerPort: 1433
                  env:
                      - name: MSSQL_PID
                        value: "Express"
                      - name: ACCEPT_EULA
                        value: "Y"
                      - name: SA_PASSWORD
                        valueFrom:
                            secretKeyRef:
                                name: mssql
                                key: SA_PASSWORD
                  volumeMounts:
                      - mountPath: /var/opt/mssql/data
                        name: mssqldb
            volumes:
                - name: mssqldb
                  persistentVolumeClaim:
                      claimName: mssql-claim
---
apiVersion: v1
kind: Service
metadata:
    name: mssql-clusterip-srv
spec:
    type: ClusterIP
    selector:
        app: mssql
    ports:
        - name: mssql
          protocol: TCP
          port: 1433
          targetPort: 1433
---
apiVersion: v1
kind: Service
metadata:
    name: mssql-loadbalancer
spec:
    type: LoadBalancer
    selector:
        app: mssql
    ports:        
        - protocol: TCP
          port: 1433
          targetPort: 1433

As stated in the beginning - I cannot, for the love of all, get into my blazor app from browser - I tried

• scp.com
• scp.com:8080
• scp.com:5000
• scp.com:5001
• scp.com:5005

Nothing seems to work, so there must be something wrong with my config files and I have no idea what could it be.

And just to be precise, additional details: Docker desktop is running along with Kubernetes

kubectl get ingress output:

NAME	CLASS	HOSTS	ADDRESS	PORTS	AGE
ingress-srv		scp.com	localhost	80	3h58m

kubectl get pods output:

NAME	READY	STATUS	RESTARTS	AGE
mssql-depl-7f46b5c696-nc6cw	1/1	Running	0	157m
scanapp-depl-76f56bc6df-nstfd	1/1	Running	0	73m

kubectl get deployments output

NAME	READY	UP-TO-DATE	AVAILABLE	AGE
mssql-depl	1/1	1	1	171m
scanapp-depl	1/1	1	1	75m

kubectl get services output:

NAME	TYPE	CLUSTER-IP	EXTERNAL-IP	PORT(S)	AGE
kubernetes	ClusterIP	10.96.0.1		443/TCP	28d
mssql-clusterip-srv	ClusterIP	10.111.202.104		1433/TCP	3h
mssql-loadbalancer	LoadBalancer	10.110.249.54	localhost	1433:30786/TCP	3h
scanapp-clusterip-srv	ClusterIP	10.97.75.150		8080/TCP,8081/TCP,5000/TCP,5001/TCP,5005/TCP	4h33m
scanappnpservice-srv	NodePort	10.103.33.184		8080:32119/TCP	4h46m

Answer 1

First, you don't need a NodePort service, just a ClusterIP . Nodeports are used if you want to access the service directly without going through the ingress controller.

In case you want to test this with a NodePort , you will need to define one, eg

- name: ui
  port: 8080
  targetPort: 80
  nodePort: 38080  # needs to be higher than 30000

But looking at your ingress rule it seems a bit off. I would try the following:

spec:
  rules:
    - host: scp.com
      http:
        paths:
          - path: /
            backend:
              serviceName: scanapp-clusterip-srv
              servicePort: ui

Answer 2

I solved the problem, or at least I think so.
In addition to @moonkotte suggestion to add the ingressClassName: nginx to ingress-srv.yaml I also changed the ingress port configuration so that it points to port 80 now.

Thanks to those changes using scp.com now correctly opens my app. Also, using NodePort access I can visit my app using localhost:30080 , where the 30080 port was set automatically (I removed the nodePort configuration line from scanapp-np-srv.yaml )

Why does the port in ingress-srv.yaml have to be set to 80 if clusterIp configuration states to set port 8080 to target port 80 - I don't know, I do not fully understand the inner workings of Kubernetes configuration - All explanations are more than welcome.

Current state of main configuration files:

ingress-srv.yaml :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-srv
  annotations:  
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "affinity"
    nginx.ingress.kubernetes.io/session-cookie-expires: "14400"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "14400"
spec:
  ingressClassName: nginx
  rules:
      - host: scp.com
        http:
            paths:
            - path: /
              pathType: Prefix
              backend:
                service:
                  name: scanapp-clusterip-srv
                  port:
                    number: 80

scanapp-np-srv.yaml :

apiVersion: v1
kind: Service
metadata:
    name: scanappnpservice-srv
spec:
    type: NodePort
    selector:
        app: scanapp
    ports:    
        - name: ui
          port: 8080
          targetPort: 80
        - name: scanapp0
          protocol: TCP
          port: 5000
          targetPort: 5000
        - name: scanapp1
          protocol: TCP
          port: 5001
          targetPort: 5001
        - name: scanapp5
          protocol: TCP
          port: 5005
          targetPort: 5005

scanapp-depl.yaml :

apiVersion: apps/v1
kind: Deployment
metadata:
    name: scanapp-depl
spec:
    replicas: 1
    selector:
        matchLabels:
            app: scanapp
    template:
        metadata:
            labels:
                app: scanapp
        spec:
            containers:
                - name: scanapp
                  image: scanapp:1.0
---
apiVersion: v1
kind: Service
metadata:
    name: scanapp-clusterip-srv
spec:
    type: ClusterIP
    selector:
        app: scanapp
    ports:
        - name: ui
          protocol: TCP
          port: 8080
          targetPort: 80
        - name: ui2
          protocol: TCP
          port: 8081
          targetPort: 443
        - name: scanapp0
          protocol: TCP
          port: 5000
          targetPort: 5000
        - name: scanapp1
          protocol: TCP
          port: 5001
          targetPort: 5001
        - name: scanapp5
          protocol: TCP
          port: 5005
          targetPort: 5005

Rest of files remained untouched.