[英]Measure processing throughput in Kafka Streams
I have a stream in java built as (anonimized some variables and classes): 我在java中建立了一个流(匿名化一些变量和类):
Map<String, Object> props = new HashMap<>();
Properties config = new Properties();
config.put(StreamsConfig.APPLICATION_ID_CONFIG, "my-stream-processing-application");
config.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "my-kafka-broker:6667");
config.put(StreamsConfig.KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
config.put(StreamsConfig.VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
KStreamBuilder builder = new KStreamBuilder();
KStream<String, String> events = builder.stream("my-topic");
events.foreach((key, value) -> {
CustomClass instance = new CustomClass(value);
for (AnotherCustomClass anotherInstance: someIterator) {
anotherInstance(instance);
}
});
KafkaStreams streams = new KafkaStreams(builder, config);
streams.start();
for kafka 0.10.0.0
: 对于kafka
0.10.0.0
:
compile group: 'org.apache.kafka', name: 'kafka-streams', version: '0.10.0.0'
compile group: 'org.apache.kafka', name: 'kafka-clients', version: '0.10.0.0'
My question is the following: 我的问题如下:
KafkaStreams streams.metrics
inside the foreach
loop? foreach
循环中访问KafkaStreams streams.metrics
? In order to read and / or print the proccesed messages throughput anotherInstance(instance)
has been evaluated anotherInstance(instance)
Kafka Streams exposes all metrics via JMX (Java Management Extensions) . Kafka Streams通过JMX(Java管理扩展)公开了所有指标。 You can check those metrics by using JConsole or VisualVM .
您可以使用JConsole或VisualVM检查这些指标。 With those tools you can explore all metrics and graph them.
借助这些工具,您可以浏览所有指标并绘制图形。
In order to check how many messages your application is processing have a look at that metric: 为了检查您的应用程序正在处理多少消息,请查看该指标:
MBean: kafka.streams:type=stream-metrics,thread.client-id=[threadId]
Attribute: process-rate
It tells you average number of processed messages per second across all tasks. 它告诉您所有任务每秒平均处理的消息数。
Full list of Kafka Streams metrics can be found in official documentation. Kafka Streams指标的完整列表可以在官方文档中找到。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.