[英]How to get consumer Kafka lag in java
I have a producer in java and consumer in nodeJS. 我有一个Java生产者和一个nodeJS消费者。 I want to know in java what is the consumer lag, so i know if i can produce more data to the topic.
我想在Java中知道什么是消费者滞后,所以我知道我是否可以针对该主题生成更多数据。
What is the API in java to get the consumer lag? Java中有什么API会导致消费者滞后?
Why do you need to know the consumer lag ? 为什么您需要知道消费者的滞后? The aim of a broker is to produce messages asynchronously.
代理的目的是异步生成消息。 If you need to have a synchronous processing, use a basic rest processing.
如果需要进行同步处理,请使用基本的剩余处理。
The actual class you can call from Java is the kafka.admin.ConsumerGroupCommand
. 您可以从Java调用的实际类是
kafka.admin.ConsumerGroupCommand
。 It's Scala code, but it's easy to call directly from Java. 这是Scala代码,但是直接从Java调用很容易。
Here's a link to the Github code for perusal: https://github.com/apache/kafka/blob/bf237fa7c576bd141d78fdea9f17f65ea269c290/core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala 这是Github代码的链接,供您细读: https : //github.com/apache/kafka/blob/bf237fa7c576bd141d78fdea9f17f65ea269c290/core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala
And here's the help output if you don't pass in any parameters. 如果您不输入任何参数,这是帮助输出。 That should help out with the commands you need to send.
这应该可以帮助您发送所需的命令。
List all consumer groups, describe a consumer group, delete consumer group info, or reset consumer group offsets. Option Description ------ ----------- --all-topics Consider all topics assigned to a group in the `reset-offsets` process. --bootstrap-server --by-duration Reset offsets to offset by duration from current timestamp. Format: 'PnDTnHnMnS' --command-config passed to Admin Client and Consumer. --delete Pass in groups to delete topic partition offsets and ownership information over the entire consumer group. For instance --group g1 -- group g2 --describe Describe consumer group and list offset lag (number of messages not yet processed) related to given group. --dry-run Only show results without executing changes on Consumer Groups. Supported operations: reset-offsets. --execute Execute operation. Supported operations: reset-offsets. --export Export operation execution to a CSV file. Supported operations: reset- offsets. --from-file Reset offsets to values defined in CSV file. --group The consumer group we wish to act on. --list List all consumer groups. --members Describe members of the group. This option may be used with '--describe' and '--bootstrap-server' options only. Example: --bootstrap-server localhost: 9092 --describe --group group1 -- members --offsets Describe the group and list all topic partitions in the group along with their offset lag. This is the default sub-action of and may be used with '--describe' and '-- bootstrap-server' options only. Example: --bootstrap-server localhost: 9092 --describe --group group1 -- offsets --reset-offsets Reset offsets of consumer group. Supports one consumer group at the time, and instances should be inactive Has 2 execution options: --dry-run (the default) to plan which offsets to reset, and --execute to update the offsets. Additionally, the -- export option is used to export the results to a CSV format. You must choose one of the following reset specifications: --to-datetime, --by-period, --to-earliest, --to- latest, --shift-by, --from-file, -- to-current. To define the scope use --all-topics or --topic. One scope must be specified unless you use '--from- file'. --shift-by Reset offsets shifting current offset by 'n', where 'n' can be positive or negative. --state Describe the group state. This option may be used with '--describe' and '-- bootstrap-server' options only. Example: --bootstrap-server localhost: 9092 --describe --group group1 -- state --timeout The timeout that can be set for some use cases. For example, it can be used when describing the group to specify the maximum amount of time in milliseconds to wait before the group stabilizes (when the group is just created, or is going through some changes). (default: 5000) --to-current Reset offsets to current offset. --to-datetime Reset offsets to offset from datetime. Format: 'YYYY-MM-DDTHH:mm:SS.sss' --to-earliest Reset offsets to earliest offset. --to-latest Reset offsets to latest offset. --to-offset Reset offsets to a specific offset. --topic The topic whose consumer group information should be deleted or topic whose should be included in the reset offset process. In `reset- offsets` case, partitions can be specified using this format: `topic1: 0,1,2`, where 0,1,2 are the partition to be included in the process. Reset-offsets also supports multiple topic inputs. --verbose Provide additional information, if any, when describing the group. This option may be used with '-- offsets'/'--members'/'--state' and '--bootstrap-server' options only. Example: --bootstrap-server localhost: 9092 --describe --group group1 -- members --verbose
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.