librdkafka线程无法在失败时清除

Question

In my application i am creating one producer and one consumer.

conf->set("dr_cb", delivery_cb, errstr);
conf->set("event_cb", event_cb, errstr);

RdKafka::Producer::create(conf.get(), errstr)

conf->set("log_level", "0", errstr);
conf->set("group.id", group_id, errstr);
conf->set("client.id", m_kafka_client_id, errstr);
conf->set("auto.offset.reset", "earliest", errstr);
conf->set("rebalance_cb", rebalance_cb, errstr);
conf->set("statistics.interval.ms", "3000", errstr);
conf->set("event_cb", event_cb, errstr);

RdKafka::KafkaConsumer::create(conf.get(), errstr)

then i try to fetch metadata as follows

err = _consumer->metadata(false, nullptr, &metadata, METADATA_TIMEOUT); 
std::unique_ptr<RdKafka::Metadata> metadata_uptr(metadata); // Handover the raw pointer to the unique_ptr now

if for some reason broker communication is not working i get error message then i producer and consumer get's deleted through unique_ptr -> destructor. and then this goes in a loop till application successfully connects to brokers.

what i have observed is i see lot's of threads getting created and remain there. at one point of time the count reached 2000 threads.

What is the correct way to cleanup Kafka?

the threads are stuck here

#0  0x00007f6acc031cf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000000004b8e65 in cnd_timedwait_ms (cnd=cnd@entry=0xab41b8, mtx=mtx@entry=0xab4190, timeout_ms=<optimized out>) at tinycthread.c:501
#2  0x00000000004853aa in rd_kafka_q_serve (rkq=0xab4190, timeout_ms=<optimized out>, max_cnt=max_cnt@entry=0, cb_type=cb_type@entry=RD_KAFKA_Q_CB_CALLBACK, 
callback=callback@entry=0x0, opaque=opaque@entry=0x0) at rdkafka_queue.c:440
#3  0x000000000045a43c in rd_kafka_thread_main (arg=arg@entry=0xabf760) at rdkafka.c:1227
#4  0x00000000004b8c07 in _thrd_wrapper_function (aArg=<optimized out>) at tinycthread.c:624
#5  0x00007f6acc02de25 in start_thread () from /lib64/libpthread.so.0
#6  0x00007f6acaa7834d in clone () from /lib64/libc.so.6

i also tried calling following routine on failure but no help...

RdKafka::wait_destroyed(5000);

Answer 1

RdKafka::KafkaConsumer was wrapped inside another container and "myobject_wrapper->close()" was not getting redirected to "RdKafka::KafkaConsumer::close"

after fixing the this issue in my application, things are working fine now.