我们如何使用 confluent-kafka-go 客户端确保可靠和低延迟的 kafka 编写？

Question

The sample producer code at https://github.com/confluentinc/confluent-kafka-go calls Producer.Produce(...) It writes the message to the sending buffer queue and return immediately. However, writing to the internal buffer queue does not guarantee successful message delivery. How can we ensure sucessful message delivery here?

for idx, word := range []string{"Welcome", "to", "the", 
                                "Confluent", "Kafka", "Golang", "client"} {
    start_time := time.Now().UnixNano()
    p.Produce(&kafka.Message{
        TopicPartition: kafka.TopicPartition{Topic: &topic, 
                                             Partition: kafka.PartitionAny},
        Value:          []byte(word),
    }, nil)
}

Answer 1

To block and send data immediately, you'd call Flush method of the producer

https://github.com/confluentinc/confluent-kafka-go/blob/master/kafka/producer.go#L335

To ensure it's written reliably, set acks=all producer config and ensure the topic is configured to have min in sync replicas as 2 or more, and topic replication factor as 3 or more