第二个节点无法绑定到 Apache Nifi 集群

Question

I have deployed two apache nifi nodes in Kubernetes. Apache nifi has statefulset kind in the yaml file.

The second node "nifi-1" raise an error, in the pod corresponding logs I have the following:

2022-12-21 18:48:16,852 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
java.io.IOException: Failed to bind to nifi-0.nifi.namespacenifi-01.svc.cluster.local/10.233.79.58:8080
 at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)
 at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310)
 at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
 at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
 at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
 at org.eclipse.jetty.server.Server.doStart(Server.java:401)
 at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
 at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:812)
 at org.apache.nifi.NiFi.<init>(NiFi.java:172)
 at org.apache.nifi.NiFi.<init>(NiFi.java:83)
 at org.apache.nifi.NiFi.main(NiFi.java:332)
Caused by: java.net.BindException: Cannot assign requested address
 at sun.nio.ch.Net.bind0(Native Method)
 at sun.nio.ch.Net.bind(Net.java:461)
 at sun.nio.ch.Net.bind(Net.java:453)
 at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
 at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
 at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
 ... 10 common frames omitted
2022-12-21 18:48:16,853 INFO [Thread-1] org.apache.nifi.NiFi Application Server shutdown started

I don't know what I missed. The Seconde node nifi-1 failed to connect. When I have one node it works well but when the second node is deployed I have the error above that I don't understand.

yaml for nifi:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nifi
  labels:
    name: nifi
    app: nifi
  annotations:
    app.kubernetes.io/name: nifi
    app.kubernetes.io/part-of: nifi
spec:
  serviceName: nifi
  replicas: 2
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: nifi
  template:
    metadata:
      labels:
        app: nifi
    spec:
      automountServiceAccountToken: false
      enableServiceLinks: false
      restartPolicy: Always
      securityContext:
        runAsGroup: 1000
        runAsUser: 1000
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: nifi
        image: XXX
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          name: nifi
        - containerPort: 8082
          name: cluster
        env:
        - name: "NIFI_SENSITIVE_PROPS_KEY"
          value: "nificluster"
        - name: NIFI_WEB_HTTP_HOST
          valueFrom:
             fieldRef:
               fieldPath: status.podIP
        - name: NIFI_WEB_HTTP_PORT
          value: "8080"
        - name: NIFI_ANALYTICS_PREDICT_ENABLED
          value: "true"
        - name: NIFI_ELECTION_MAX_CANDIDATES
          value: "2"
        - name: NIFI_ELECTION_MAX_WAIT
          value: "1 min"
        - name: NIFI_CLUSTER_IS_NODE
          value: "true"
        - name: NIFI_JVM_HEAP_INIT
          value: "3g"
        - name: NIFI_JVM_HEAP_MAX
          value: "4g"
        - name: NIFI_CLUSTER_NODE_CONNECTION_TIMEOUT
          value: "2 min"
        - name: NIFI_CLUSTER_PROTOCOL_CONNECTION_HANDSHAKE_TIMEOUT
          value: "2 min"        
        - name: NIFI_CLUSTER_NODE_PROTOCOL_MAX_THREADS
          value: "15"
        - name: NIFI_CLUSTER_NODE_PROTOCOL_PORT
          value: "8082"
        - name: NIFI_CLUSTER_NODE_READ_TIMEOUT
          value: "15"
        - name: NIFI_ZK_CONNECT_STRING
          value: "zookeeper:2181"
        - name: NIFI_CLUSTER_PROTOCOL_IS_SECURE
          value: "false"
        - name: NIFI_CLUSTER_NODE_ADDRESS
          valueFrom:
             fieldRef:
               fieldPath: status.podIP
        # - name: HOSTNAME
        #   valueFrom:
        #     fieldRef:
        #       fieldPath: status.podIP
        livenessProbe:
          exec:
            command:
              - pgrep
              - java
          initialDelaySeconds: 60
          periodSeconds: 30
          timeoutSeconds: 10
          failureThreshold: 3
          successThreshold: 1
        readinessProbe:
          exec:
            command:
              - pgrep
              - java
          initialDelaySeconds: 180
          periodSeconds: 30
          timeoutSeconds: 10
          failureThreshold: 3
          successThreshold: 1
        resources:
          requests:
            cpu: 400m
            memory: 1Gi
          limits:
            cpu: 500m
            memory: 2Gi
      volumes:
        - name: pv--01
          persistentVolumeClaim:
            claimName: pv-claim

Answer 1

Usually this error occurs when the folders pointed to by nifi.nar.library.directory.* was inaccessible, eg the Symbolic links were no longer valid due to file system changes.

As stated, the referenced location may be inaccessible, be it permissions or broken links. Correct permissions or links and service will start fine

Or Try running the below commands:

sudo chown -R nifi:nifi /opt/nifi/
sudo service nifi start