以公差匹配数组中的值

Question

I'm trying to weed out duplicate values in an array, which I'm successfully accomplishing with the "List::MoreUtils uniq/distinct" function.

However, I would also like to count those values that fall within a given tolerance, say +-5, as duplicates as well (I think tolerance is also sometimes referred to as "delta").

For example, if 588 is a value in the array, but so is 589, because the difference falls within the tolerance of 5, 589 gets the boot.

Without some nasty/costly cross-checking of arrays, is there an elegant way to do this?

EDIT: ikegami brought to my attention some ambiguity in my question and I'm having a bit of a hard time wrapping my head around the problem. However, I think I have it worked out.

[500,505,510,515,525,900]

If you try to match the values throughout the entire array, you should get:

[500,510,525,900]

It hits 505, sees it as non-unique, removes it from the array, then sees 510 as newly-unique due to the absence of 505, and so on. This, I imagine is the way I outlined my original question, but on reflection, it seems it's a useless and fairly arbitrary data set.

What I really want is the following match:

[500,900]

It represents a group of numbers that are within 5 of each other, while also spotting the vast variance in the 900 value. This seems to be more useful information than the former and it appears that perreal's answer gets me close. Sorry for the confusion, and many thanks to ikegami as well as perreal for forcing my clarification.

EDIT 2 An even better match would be:

[510,900]

510, being the median of all the sequential +-5 values.

However, I recognize that now we're deviating severely from my original question, so I would be more than happy with an answer to my EDIT 1 clarification.

Answer 1

This is a deceptively complex problem, as the data must not only be organized into groups, but also those groups must be combined if a new data point is seen that belongs to more than one of them.

This program seems to do what you need. It keeps a list of arrays @buckets , where each element contains all values seen so far that is within TOLERANCE of one other. This list is scanned to see if each value falls within range of the maximum and minimum values already present. The index of the groups that the value belongs to are stored in memberof , and there will always be zero, one or two entries in this array.

All the groups specified by @memberof are removed from @buckets , combined together with the new data value, sorted, and replaced as a new group in the list.

At the end the @buckets array is converted to a list of median values, sorted and displayed. I have used Data::Dump to show the contents of the groups before they are aggregated to their median values.

To generate your desired output 510, 900 from the list 500, 510, 525, 900 the value for TOLERANCE must be increased so that values that differ by 15 or less are combined.

use strict;
use warnings;

use constant TOLERANCE => 5;

my @data = qw/ 500 505 510 515 525 900 /;

my @buckets;

for my $item (@data) {

  my @memberof;
  for my $i (0 .. $#buckets) {
    if ($item >= $buckets[$i][0] - TOLERANCE and $item <= $buckets[$i][-1] + TOLERANCE) {
      push @memberof, $i;
    }
  }

  my @newbucket = ($item);
  for my $i (reverse @memberof) {
    push @newbucket, @{ splice @buckets, $i, 1 };
  }

  push @buckets, [ sort { $a <=> $b } @newbucket ];
}

use Data::Dump;
dd @buckets;

@buckets = sort { $a <=> $b } map median(@$_), @buckets;
print join(', ', @buckets), "\n";

sub median {

  my $n = @_;
  my $i = $n / 2;

  if ($n % 2) {
    return $_[$i];
  }
  else {
    return ($_[$i-1] + $_[$i]) / 2;
  }
}

output

([500, 505, 510, 515], [525], [900])
507.5, 525, 900

Answer 2

Isolate the samples that form a chain where each is within the tolerance of the next, then choose one from that group.

sub collapse {
   my $tol = shift;

   my @collapsed;
   while (@_) {
      my @group = shift(@_);
      while (@_ && $group[-1] + $tol >= $_[0]) {
         push @group, shift(@_);
      }

      push @collapsed, choose_from(@group);
   }

   return @collapsed;
}

say join ',', collapse(5 => 500,505,510,515,525,900);

So how do you choose? Well, you could return the average.

use List::Util qw( sum );

sub choose_from {
   return sum(@_)/@_;
}

# Outputs: 507.5,525,900

Or you could return the median.

use List::Util qw( sum );

sub choose_from {
   my $median;
   if (@_ % 2 == 0) {
      my $avg = sum(@_)/@_;
      my $diff0 = abs( $_[ @_/2 - 1 ] - $avg );
      my $diff1 = abs( $_[ @_/2 - 0 ] - $avg );
      if ($diff0 <= $diff1) {
         return $_[ @_/2 - 1 ];
      } else {
         return $_[ @_/2 - 0 ];
      }
   } else {
      return $_[ @_/2 ];
   }
}

# Outputs: 505,525,900