石墨与收集。 保留不适用

Question

I have installed collectd on BIND9 hosts and configured it to pass metrics to graphite. I want to be able to view metrics for at least 1 month but retentions in storage-schema.conf not applied for collectd section, and applied last default section that stores only last 24h metrics. I understand that I need to set correct retentions regex to match metrics but it is seems to me that is correct already but it does not work :(

My storage-schemas.conf looks like this for now:

[carbon]
pattern = ^carbon\.
retentions = 60:90d

[mxservers]
pattern = ^mx-servers\.*
retentions = 60s:7d,5m:2y 

[ns.servers]
pattern = ^ns\d\.collectd\..*
retentions = 60s:7d,10m:2y

[collectd]
pattern =^collectd\.*
retentions = 60s:7d,10m:2y 

# *** Netapp Monitoring ***
[netapp.capacity]
pattern = ^netapp\.capacity\.*
retentions = 15m:100d, 1d:5y

[netapp.poller.capacity]
pattern = ^netapp\.poller\.capacity\.*
retentions = 15m:100d, 1d:5y

[netapp.perf]
pattern = ^netapp\.perf\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

[netapp.poller.perf]
pattern = ^netapp\.poller\.perf\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

[netapp.perf7]
pattern = ^netapp\.perf7\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

[netapp.poller.perf7]
pattern = ^netapp\.poller\.perf7\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y
# *** Netapp Monitoring ***

[default_1min_for_1day]
pattern = .*
retentions = 60s:1d

I have a problem with section named - [ns.servers], tried to set regex in it to:

^ns\d\.collectd\.
^ns\d\.collectd\..*
^ns.\.collectd\.
^ns.\.collectd\..*

None of them solves the problem. The metrics are stored with names like this:

ns1.collectd.load.load.shortterm
ns2.collectd.load.load.longterm
ns1.collectd.interface-bond0.if_packets.rx
ns2.collectd.interface-bond0.if_packets.tx

Please help me to set correct regex to get it match.

Answer 1

Did you happen to configure the regexes in storage-schemas.conf after the .wsp files were created? If so you have to resize the existing .wsp files manually. You can do this with the whisper-resize utility included in the whisper package (or just delete the files and let carbon recreate them).

storage-schemas.conf is only applied when creating a new .wsp file.

Answer 2

I finally realized what the problem is! I take a look to /var/log/carbon/creates.log and found that my metrics for collecd is stored as ns2/collectd.* - "hostname"+"/"+"collectd" - because in configuration of collectd on DNS-servers there was an option Postfix "/collectd" that make this.

05/06/2017 09:54:22 :: creating database file /var/lib/graphite/whisper/ns2/collectd/cpu-1/cpu-idle.wsp (archive=[(60, 1440)] xff=None agg=None)
05/06/2017 09:54:22 :: new metric ns2/collectd.disk-sda.disk_octets.write matched schema default_1min_for_1day
05/06/2017 09:54:22 :: new metric ns2/collectd.disk-sda.disk_octets.write matched aggregation schema default

So the solution was to make regex look like this: pattern = ^ns[1-2]\\/collectd\\.* - escaped "/" - character and then I deleted old metrics and new metric files was created with correct retentions!

05/06/2017 11:32:12 :: creating database file /var/lib/graphite/whisper/ns2/collectd/df-boot-efi/df_complex-free.wsp (archive=[(60, 10080), (600, 105120)] xff=None agg=None)
05/06/2017 11:32:12 :: new metric ns2/collectd.interface-eth0.if_errors.tx matched schema ns.servers
05/06/2017 11:32:12 :: new metric ns2/collectd.interface-eth0.if_errors.tx matched aggregation schema default