I'm trying to monitor Apache Ignite with Prometheus' JMX exporter, but I'm seeing only default JVM metrics plus metrics only for "Thread Pools" Beans. JMX exporter run as agent:
/usr/bin/java -XX:+AggressiveOpts -javaagent:/etc/prometheus/jmx_prometheus_javaagent-0.13.0.jar=8080:/etc/prometheus/prometheus_config.yml -Xms1g -Xmx1g -server -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8 -Dcom.sun.management.jmxremote.rmi.port=49112 -Djava.rmi.server.hostname=127.0.0.1 -DIGNITE_QUIET=true -DIGNITE_SUCCESS_FILE=/usr/share/apache-ignite/work/ignite_success_ed3b2798-4d48-4188-94ac-1728fa8628dc -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49112 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -DIGNITE_HOME=/usr/share/apache-ignite -DIGNITE_PROG_NAME=/usr/share/apache-ignite/bin/ignite.sh -cp /usr/share/apache-ignite/libs/*:/usr/share/apache-ignite/libs/ignite-indexing/*:/usr/share/apache-ignite/libs/ignite-spring/*:/usr/share/apache-ignite/libs/licenses/* org.apache.ignite.startup.cmdline.CommandLineStartup /etc/apache-ignite/default-config.xml
Ignite config has enables metrics:
<property name="metricExporterSpi">
<list>
<bean class="org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi"/>
</list>
</property>
I can see a lot of ignite metrics from jconsole connected to 49112 port.
Tried different jmx-exporter options, nothing helped.
---
hostPort: 127.0.0.1:49112
lowercaseOutputLabelNames: true
lowercaseOutputName: true
---
lowercaseOutputLabelNames: true
lowercaseOutputName: true
rules:
- pattern: "^org.apache<clsLdr=(.+), name=sys"
name: ignite_sys_stats
help: Ignite cluster amount of heap memory in bytes
labels:
attr: $3
type: GAUGE
and even empty config, that should mean "gather everything as is", still see just standard JVM + "Thread Pools". Can you suggest what's wrong here?
Create an empty prometheus_config.yml file. Make sure the file is properly read. Trace here: https://github.com/prometheus/jmx_exporter/blob/ce04b7dca8615d724d8f447fa25c44ae1c29238b/collector/src/main/java/io/prometheus/jmx/JmxCollector.java#L75 to make sure you are using the correct file.
remove the metricExporterSpi property
connect to the port specified, 8080 in your case, to see all the results.
Your pattern matching rules are not allowing Apache Ignite results to show properly.
You can use debug/tracing instructions here: https://github.com/prometheus/jmx_exporter to see what is happening.
pattern rule processing is here: https://github.com/prometheus/jmx_exporter/blob/ce04b7dca8615d724d8f447fa25c44ae1c29238b/collector/src/main/java/io/prometheus/jmx/JmxCollector.java#L357
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.