![](/img/trans.png)
[英]How to consume from latest offset using Sarama Go Kafka Consumer
[英]Unable to consume messages from locally running Kafka server, using Golang Sarama Package
我正在制作一個簡單的Telegram機器人,該機器人將從本地Kafka服務器讀取消息並將其打印出來進行聊天。 zookeeper和kafka服務器配置文件均為默認設置。 控制台使用者作品。 當我嘗試使用Golang Sarama包從代碼中吸收消息時,問題就會出現。 在添加這些行之前:
case err := <-pc.Errors(): log.Panic(err)
該程序僅將消息打印一次,之后它將停止。 現在,它會在將kafka: error while consuming test1/0: kafka: broker not connected
地將其列為日志: kafka: error while consuming test1/0: kafka: broker not connected
這是代碼:
type kafkaResponse struct {
telega *tgbotapi.Message
message []byte
}
type kafkaRequest struct {
telega *tgbotapi.Message
topic string
}
var kafkaBrokers = []string{"localhost:9092"}
func main() {
//channels for request response
var reqChan = make(chan kafkaRequest)
var respChan = make(chan kafkaResponse)
//starting kafka client routine to listen to topic channnel
go consumer(reqChan, respChan, kafkaBrokers)
//bot thingy here
bot, err := tgbotapi.NewBotAPI(token)
if err != nil {
log.Panic(err)
}
bot.Debug = true
log.Printf("Authorized on account %s", bot.Self.UserName)
u := tgbotapi.NewUpdate(0)
u.Timeout = 60
updates, err := bot.GetUpdatesChan(u)
for {
select {
case update := <-updates:
if update.Message == nil {
continue
}
switch update.Message.Text {
case "Topic: test1":
topic := "test1"
reqChan <- kafkaRequest{update.Message, topic}
}
case response := <-respChan:
bot.Send(tgbotapi.NewMessage(response.telega.Chat.ID, string(response.message)))
}
}
這是consumer.go:
func consumer(reqChan chan kafkaRequest, respChan chan kafkaResponse, brokers []string) {
config := sarama.NewConfig()
config.Consumer.Return.Errors = true
// Create new consumer
consumer, err := sarama.NewConsumer(brokers, config)
if err != nil {
panic(err)
}
defer func() {
if err := consumer.Close(); err != nil {
panic(err)
}
}()
select {
case request := <-reqChan:
//get all partitions on the given topic
partitionList, err := consumer.Partitions(request.topic)
if err != nil {
fmt.Println("Error retrieving partitionList ", err)
}
initialOffset := sarama.OffsetOldest
for _, partition := range partitionList {
pc, _ := consumer.ConsumePartition(request.topic, partition, initialOffset)
go func(pc sarama.PartitionConsumer) {
for {
select {
case message := <-pc.Messages():
respChan <- kafkaResponse{request.telega, message.Value}
case err := <-pc.Errors():
log.Panic(err)
}
}
}(pc)
}
}
}
在代碼中設置所有PartitionConsumer
之后,您將關閉消費者
defer func() {
if err := consumer.Close(); err != nil {
panic(err)
}
}()
但是,文檔指定僅在所有PartitionConsumers已關閉之后才關閉使用者。
// Close shuts down the consumer. It must be called after all child
// PartitionConsumers have already been closed.
Close() error
我建議您將sync.WaitGroup
添加到功能go func(pc sarama.PartitionConsumer) {
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.