Kafka Producer 連接不上服務器，不會拋出任何異常

Question

我有一個非常簡單的設置來向 Kafka 發送消息：

        var producerConfig = new ProducerConfig
        {
            BootstrapServers = "www.example.com",
            SecurityProtocol = SecurityProtocol.SaslSsl,
            SaslMechanism = SaslMechanism.ScramSha512,
            SaslUsername = _options.SaslUsername,
            SaslPassword = _options.SaslPassword,
            MessageTimeoutMs = 1
        };

        var producerBuilder = new ProducerBuilder<Null, string>(producerConfig);

        using var producer = producerBuilder.Build();

        producer.Produce("Some Topic", new Message<Null, string>()
        {
            Timestamp = Timestamp.Default,
            Value = "hello" 
        });

之前，這段代碼運行良好。 今天它決定停止工作，我正試圖找出原因。 我試圖讓生產者在未能傳遞消息時拋出異常，但它似乎永遠不會崩潰。 即使我填寫了錯誤的用戶名和密碼，生產者仍然不會崩潰。 在我的本地 output window 中甚至沒有日志行。當生產者從未顯示任何問題時，我如何調試我的 Kafka 連接？

Answer 1

您可以將SetErrorHandler()添加到 ProducerBuilder。 它看起來像這樣：

    var producerBuilder = new ProducerBuilder<Null, string>(producerConfig)
                              .SetErrorHandler(errorMessageString => .....);

在該 lambda 中設置一個斷點，您可以在出現錯誤時中斷。

Answer 2

生產是異步的，不阻塞，function 簽名是

void Produce(string topic, Message<TKey, TValue> message, Action<DeliveryReport<TKey, TValue>> deliveryHandler = null)

為了驗證消息是否已正確傳遞

您可以添加一個交付報告處理程序 function 例如

 private void DeliveryReportHandler(DeliveryReport<int, T> deliveryReport)
        {
            if (deliveryReport.Status == PersistenceStatus.NotPersisted)
            {
                _logger.LogError($"Failed message delivery: error reason:{deliveryReport.Error?.Reason}");
                _messageWasNotDelivered = true;
            }
        }

_messageWasNotDelivered = false;
_producer.Produce(topic,
                   new Message<int, T>
                   {
                       Key = key,
                       Value = entity
                   },
                   DeliveryReportHandler)
_producer.Flush(); // Wait until all outstanding produce requests and delivery report callbacks are completed

if(_messageWasNotDelivered ){
     // handle non delivery
}

可以像這樣簡單地調整此代碼以進行批量生產

_messageWasNotDelivered = false;
foreach(var entity in entities){
    _producer.Produce(topic,
                   new Message<int, T>
                   {
                       Key = entity.Id,
                       Value = entity
                   },
                   DeliveryReportHandler)
}

_producer.Flush(); // Wait until all outstanding produce requests and delivery report callbacks are completed

if(_messageWasNotDelivered ){
     // handle non delivery
}