Kubernetes MirrorMaker2 Cannot Load Secret

Question

I have a weird issue that no one can pinpoint. To make sure it was not an Azure Kube.netes issue, I also spun up minikube to test locally and I am getting the same error. The one thing in common Strimzi 0.28 for MirrorMaker2.

You can read the entire thread here in case it might help. We are stuck on a dead end. The link to the entire discussion is github under strimzi;

• Strimzi Mirrormaker2 and Event Hub Odd Error, Can't Pinpoint .

I moved it as I didn't want to spam as a gentleman by the name of scholzj helped and gave some great advice. But nothing seems to work.

Here is what I have done.

Create The Secret

Replaced actual data with <name>, <keyName>, <password> for posting purposes.

cat <<EOF | kubectl apply -n kafka-cloud -f - 
apiVersion: v1
kind: Secret
metadata:
  name: eventhubssecret
  namespace: kafka-cloud
type: Opaque
stringData:
  eventhubspassword: Endpoint=sb://<name>.servicebus.windows.net/;SharedAccessKeyName=<keyName>;SharedAccessKey=<password>
EOF

Validated and shows the values exist in the namespace "kafka-cloud".

Run Mirror Maker 2

cat <<EOF | kubectl apply -n kafka-cloud -f -
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
  name: mirror-maker-eventhub
spec:
  version: 3.1.0
  replicas: 1
  connectCluster: "eventhub"
  clusters:
  - alias: "my-kafka-cluster"
    bootstrapServers: my-cluster-kafka-bootstrap:9092
  - alias: "eventhub"
    bootstrapServers: <name>.servicebus.windows.net:9093
    config:
      config.storage.replication.factor: 1
      offset.storage.replication.factor: 1
      status.storage.replication.factor: 1
      producer.connections.max.idle.ms: 180000
      producer.metadata.max.age.ms: 180000
    authentication: 
      type: plain 
      username: $ConnectionString
      passwordSecret: 
        secretName: eventhubssecret 
        password: eventhubspassword
    tls:
      trustedCertificates: []
  mirrors:
  - sourceCluster: "my-kafka-cluster"
    targetCluster: "eventhub"
    sourceConnector:
      config:
        replication.factor: 1
        offset-syncs.topic.replication.factor: 1
        sync.topic.acls.enabled: "false"
    heartbeatConnector:
      config:
        heartbeats.topic.replication.factor: 1
    checkpointConnector:
      config:
        checkpoints.topic.replication.factor: 1
    topicsPattern: ".*"
    groupsPattern: ".*"
EOF

It gets created;

• kafkamirrormaker2.kafka.strimzi.io/mirror-maker-eventhub created

Then on the describe we get this in both azure and also minikube on a different machine. The common denominator is strimzi 0.28 v1beta2. All the examples I have seen with azure and mirrormaker2 are older. But we can't figure out why the secret is not passed??

Status:
  Conditions:
    Last Transition Time:  2022-03-17T00:13:53.909810Z
    Message:               PLAIN authentication selected, but username or password configuration is missing.
    Reason:                InvalidResourceException
    Status:                True
    Type:                  NotReady
  Observed Generation:     1
  Replicas:                0
Events:                    <none>

Has anyone seen this behavior before, if so what am I missing? Or is this a bug with strimzi and Mirrormaker2? I just need to test moving data from Kafka to Event Hub and thought MirrorMaker2 was the way to go. But after two days now, I am having my doubts.

Thanks.

Answer 1

The issue was using cat <<EOF | kubectl apply -n kafka-cloud -f -

I think it's because of $ in the username. EH needs this as the actual username for the connection. Once I made the above into a file between cat <<EOF and the last EOF it ran from the CLI without changing anything.

It worked.

kubectl apply -n kafka-cloud -f fileName.yaml

When working with EH use a file, don't cat <<EOF it in.