为什么 kafka 生产者在第一条消息上很慢？

Question

I am using kafka producer to send prices to topic. When I send first message it prints producer config and then send message due to this it takes more time to send first message .

After first message it tooks hardly 1/2 milliseconds to send a message .

My question is can we do something so that configuration part will skip or we can start before to send first message ?

I am using spring kafka into my project. I read other question also but not really helpful .

Application.yml

server:
  port: 8081
spring:
    kafka:
      bootstrap-servers:   ***.***.*.***:9092
      producer:
          key-serializer: org.apache.kafka.common.serialization.StringSerializer
          value-serializer: org.apache.kafka.common.serialization.StringSerializer
      

Producer Values :

acks = 1
batch.size = 16384
bootstrap.servers = [192.168.1.190:9092]
buffer.memory = 33554432
client.dns.lookup = default
client.id = 
compression.type = none
connections.max.idle.ms = 540000
delivery.timeout.ms = 120000
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.StringSerializer

I refer following questions but not helped.

1. Why is camel kafka producer very slow?
2. Kafka producer is slow on first message

Answer 1

During the first invocation of the KafkaProducer.send method, the Kafka producer fetches the partition metadata for the topic. Fetching the metadata blocks the send method from returning immediately. The Kafka producer caches the metadata, so subsequent sends are much faster. The Kafka producer caches the metadata for metadata.max.age.ms (default 5 minutes), after which it again fetches the metadata to proactively discover any new brokers or partitions.

When your application starts, you could invoke the KafkaProducer.partitionsFor method to fetch and cache the metadata, but when the cache expires after 5 minutes, the next send will be slow because it fetches the metadata again. If your Kafka environment is static, that is, new brokers and partitions are not created while your application is running, then consider configuring metadata.max.age.ms to a very long time duration, so the metadata is kept in the cache longer.

Answer 2

what do mean by first message is slow? kafka send is asynchronous and should not hinder your own performance. it uses a buffer that it uses to send records in batches.

read below java docs for more info,

https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html