integer 与 prometheus/grafana 中向量集的区别

Question

I can't realize how to figure out with metrics nss_chan_last_seq and nss_chan_subs_last_sent . I need to get difference between max(last_seq) and min(last_sent) for every durable_name So i have metric last_seq:

max(nss_chan_last_seq{channel="some-channel", instance=~"some-cluster-cl.*"}) by (channel)

And last_sent:

min(nss_chan_subs_last_sent{channel="some-channel", durable_name=~"durable-name-s.*", durable_name!~".*test"}) by (channel, durable_name)

And when I put something like this:

max(nss_chan_last_seq{channel="some-channel", instance=~"some-cluster-cl.*"}) by (channel) - on (channel) min(nss_chan_subs_last_sent{channel="some-channel", durable_name=~"durable-name-s-.*", durable_name!~".*test"}) by (channel, durable_name) 

I got this error:

execution: found duplicate series for the match group {channel="some-channel"} on the right hand-side of the operation: [{channel="some-channel", durable_name="durable-name-s16"}, {channel="some-channel", durable_name="durable-name-s14"}];many-to-many matching not allowed: matching labels must be unique on one side

Can someone please explain me, how to figure out with this?

I tried to play with on(), group_left(), and group_left() ok with one vector in nss_chan_last_seq , but i have several durable_names for one channel

Answer 1

The problem with the query is that the instant vector from the left-hand side operand has multiple matches on the right-hand side instant vector operand.

In such cases, you must explicitly tell Prometheus how do you want to match the time series.

Group Modifiers (group_left, group_right)

Group modifiers enable many-to-one (group_left) / one-to-many (group_right) vector matching.

To make sure all the "many side" labels are preserved, the result has all the labels from samples on the operand on this "many side", which is the left side for group_left and the right side for group_right .

So:

• if the "many side" is a left-hand side operand --> use group_left .

• if the "many side" is a right-hand side operand --> use group_right .

---

Let's take your operands.

Left -hand side operand:

max(nss_chan_last_seq{channel="some-channel", instance=~"some-cluster-cl.*"}) by (channel)

{channel="fizz"} 1
{channel="bazz"} 2

Right -hand side operand:

min(nss_chan_subs_last_sent{channel="some-channel", durable_name=~".*"}) by (channel, durable_name)

{channel="fizz", durable_name="foo"} 10
{channel="bazz", durable_name="foo"} 20

{channel="fizz", durable_name="bar"} 30
{channel="bazz", durable_name="bar"} 40

The "many side" is the right -hand side operand, so group_right should be used:

max(nss_chan_last_seq{channel="some-channel", instance=~"some-cluster-cl.*"}) by (channel)
- on (channel)
min(nss_chan_subs_last_sent{channel="some-channel", durable_name=~".*"}) by (channel, durable_name)