如何在Prometheus中绘制各个Summary指标实例的图？

Question

I'm using Prometheus' Summary metric to collect the latency of an API call. Instead of making an actual API call, I'm simply calling Thread.sleep(1000) to simulate a 1 second api-call latency value -- this makes the Summary hold a value of .01 (for 1 second of latency). But if, for example, I invoke Thread.sleep(1000) twice in the same minute, the Summary metric ends up with a value of .02 (for 2 seconds of latency), instead of two individual instances of .01 latency that just happened to occur within the same minute. My problem is the Prometheus query. The Prometheus query I am currently using is: rate(my_custom_summary_sum[1m]) .

What should my Prometheus query be, such that I can see the latency of each individual Thread.sleep(1000) invocation . As of right now, the Summary metric collects and displays the total latency sum per minute. How can I display the latency of each individual call to Thread.sleep(1000) (ie the API request)?

private static final Summary mySummary = Summary.build()
                                        .name("my_custom_summary")
                                        .help("This is a custom summary that keeps track of latency")
                                        .register();

Summary.Timer requestTimer = mySummary.startTimer(); //starting timer for mySummary 'Summary' metric
        Thread.sleep(1000); //sleep for one second
        requestTimer.observeDuration(); //record the time elapsed

This is the graph that results from this query: Prometheus graph

Answer 1

Prometheus is a metrics-based monitoring system, it cares about overall performance and behaviour - not individual requests.

What you are looking for is a logs-based system, such as Graylog or the ELK stack.