Producer/consumer JMX metrics

Question

I am stuck in a dump problem. It seems at least like this...

I am trying to generate JMX metrics from kafka console producer right now. I am done with broker.

I have a collectd conf file

LoadPlugin java
<Plugin java>
    <Plugin "GenericJMX">
        <MBean "kafka.producer.request-rate">
            ObjectName "kafka.producer:type=producer-metrics,clientId=([-.\w]+)"
            InstancePrefix "all"
            <Value>
                InstancePrefix "kafka-producer-request-rate"
                Type "gauge"
                Table false
                Attribute "request-rate"
            </Value>
        </MBean>
        <Connection>
            ServiceURL "service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi"
            Host "tgarg"
            Collect "classes"
            Collect "garbage_collector"
            Collect "memory-heap"
            Collect "memory-nonheap"
            Collect "memory_pool"
            Collect "kafka.producer.request-rate"
        </Connection>
    </Plugin>
</Plugin>

This looks good to me but on collectd end I am getting

collectd[552]: GenericJMXConfMBean: No MBean matched the ObjectName kafka.producer:type=producer-metrics,clientId=([-.w]+)

Any help is appreciated.
Thanks-Tarun

Answer 1

Do you see jmx enabled when you run jps -v ?

-Dcom.sun.management.jmxremote.port=9999 should be visible in your case.

More information on how to configure jmx : http://docs.confluent.io/current/cp-docker-images/docs/operations/monitoring.html

Answer 2

You have to enable JMX setting JMX_PORT environment variable before calling kafka_console_producer.sh. Something like this:

JMX_PORT=9999 ./kafka-console-producer.sh --broker-list 127.0.0.1:9092 --topic topic1

See bin/kafka-run-class.sh script for details.

Answer 3

As i said this is dump problem. collectD doesn't understand ([-.\\w]+), i replaced this with * and clientId with client-id (my bad).

Thanks all