External access to Kafka using Strimzi

Question

I'm attempting to provide bi-direction external access to Kafka using Strimzi by following this guide: Red Hat Developer - Kafka in Kubernetes

My YAML taken from the Strimizi examples on GitHub , is as follows:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.0.0
    replicas: 1 #3
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: loadbalancer
        tls: false
        configuration:
          #externalTrafficPolicy: Local
          #loadBalancerSourceRanges:
          #  - 10.0.0.200/32
          brokers:
          - broker: 0
            advertisedHost: 10.0.0.200
            advertisedPort: 30123
    config:
      offsets.topic.replication.factor: 1 #3
      transaction.state.log.replication.factor: 1 #3
      transaction.state.log.min.isr: 1 #2
      log.message.format.version: "3.0"
      inter.broker.protocol.version: "3.0"
    storage:
      type: ephemeral
  zookeeper:
    replicas: 1 #3
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

When running kubectl get services I'm presented with the following:

NAME                                  TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
kubernetes                            ClusterIP      10.96.0.1       <none>        443/TCP                               48m
my-cluster-kafka-0                    LoadBalancer   10.107.190.96   <pending>     9094:31964/TCP                        29m
my-cluster-kafka-bootstrap            ClusterIP      10.99.34.246    <none>        9091/TCP,9092/TCP,9093/TCP            43m
my-cluster-kafka-brokers              ClusterIP      None            <none>        9090/TCP,9091/TCP,9092/TCP,9093/TCP   43m
my-cluster-kafka-external-bootstrap   LoadBalancer   10.99.91.68     <pending>     9094:31442/TCP                        29m
my-cluster-zookeeper-client           ClusterIP      10.101.216.35   <none>        2181/TCP                              45m
my-cluster-zookeeper-nodes            ClusterIP      None            <none>        2181/TCP,2888/TCP,3888/TCP            45m

Note the my-cluster-kafka-0 and my-cluster-kafka-external-bootstrap has a <pending> EXTERNAL-IP . What am I missing within my YAML file to provide bi-direction external access to my-cluster-kafka-0 ?

Answer 1

Strimzi just created the Kubernetes Service of type Loadbalancer . It is up to your Kubernetes cluster to provision the load balancer and set its external address which Strimzi can use. When the external address is listed as pending it means the load balancer is not (yet) created. In some public clouds that can take few minutes, so it might be just about waiting for it. But keep in mind that the load balancers are not supported in all environments => and when they are not supported, you cannot really use them. So you really need to double check whether your environment supports them or not. Typically, different clouds would support load balancers while some local or bare-metal environments might not (but it really depends).

I'm also not really sure why did you configured the advertised host and port:

            advertisedHost: 10.0.0.200
            advertisedPort: 30123

When using load balancers (assuming they would be supported in your environments), you would normally want to use the loadbalancer address which will be automatically set as the advertised host / port. Apart from that, your YAML looks good, but the loadbalancer support might be missing.