find the max one number in a list of double, need to care about precision?

Question

Suppose have a vector, and need to find out the max number. My colleague told me that double numbers must be treated in a special way to do this, his code is like:

double max = v[0];
for (int i = 0; i < v.size(); ++i) {
   if (compare(max, v[i]) < 0) max = v[i];
}

int compare(double a, double b) {
    double z = a - b;
    if (z < -0.000001) return -1;
    if (z > 0.000001) return 1;
    return 0;
}

I don't think it needs to be that complex, simply using '<' or '>' can do the job, it shouldn't care about equals. But my colleague insists that compare with epsilon is must for finding out the max number. Is that true?

Answer 1

For the purpose of finding the largest number, you're right, a simple < suffices.

What your colleague is thinking of (we hope, anyway) is dealing with equality of floating point numbers. For example, if you're computing what should be the same value in two different ways, you might easily see some minute difference between the two--even (a+b)+c vs. a+(b+c) can change a result (even though from a mathematical viewpoint, the two should be identical).

When you do this, however, you normally want to scale the difference you allow between the numbers based on the magnitude of the numbers.

For example, let's consider a double that can represent about 15 significant digits.

If your numbers are around 1e+200, then the smallest difference between those two numbers that can be represented is approximately 1e+185. Asking whether the difference is smaller than 0.000001 is pointless--either the results are identical, or else the difference is much larger than that.

Contrariwise, if your numbers were around 1e-200, then the smallest difference between them that could be represented would be around 1e-215. A difference of 0.000001 is (again) utterly ridiculous to contemplate--it could only happen if one of the two calculations was incorrect by ~196 orders of magnitude ¹ .

So, to do this, you pick a number of places after the decimal point that need to match for you to consider the two equal, and multiply that by the numbers to get a maximum delta. For example, if you decide they need to agree to 7 decimal places, and the numbers are in the range of 1eN, then the maximum difference is 1e(N-7). If N is 100, then the maximum delta is 1e93. If N is -150, then the maximum delta is 1e-157.

This still needs to be used with considerable care, especially when dealing with groups of numbers. The problem is that with an approximately equal like this is no longer transitive. Even if the numbers are in a range where an Epsilon of 0.000001 might make some sense, it can say that A == B, and B == C, but A != C. This can lead to quite surprising results, to put it mildly (in some cases like sorting, can lead to complete failure, because you're violated the requirement for a strict weak ordering).

As far as looking at your original problem of finding the maximum value in a vector goes, this basically means that just using < will find the value that's actually the largest. Depending on roundoff error, however, there may be other values that are smaller, but should theoretically be larger. Depending on how those results were calculated and what you're trying to accomplish, you might want to consider finding not just the single value that's the largest, but all the other values that are within a chosen maximum error of the largest. The others would be values that aren't quite as large, but are close enough that they could represent the largest measurement (or whatever exactly you're working with).

There's one other point that's probably worth mentioning: some floating point formats include representations that are "not a number". This will produce false for every possible comparison (in fact, a common way of detecting a NaN is if (x != x) /* it's a NaN */ ). As such, if your input values might include a NaN, seemingly identical comparisons could give entirely different results. For example, if x < y and if not y >= x should normally be the same, but if either x or y is a NaN, they won't be.

---

1. To try to put 196 orders of magnitude in perspective, let's assume you were doing calculations attempting to compare the size of a neutron with the size of a proton. Then you decide to check whether the difference you got between those two sizes was greater than the diameter of the Milky Way galaxy.

   Oh, but that wouldn't be 196 orders of magnitude. That's only about 36 orders of magnitude. So let's check if the difference we got was larger than the (currently believed) diameter of the universe. That gets us up to around 50 orders of magnitude.

   I guess I've kind of failed at putting it into perspective though--even if we look at the smallest size that string theory attributes to a "string" and compare that to the (generally accepted) size of the universe, the two aren't even close to 196 orders of magnitude apart. Worse, I doubt the size of a string or the size of the known universe is something anybody can really visualize meaningfully anyway. The one is too small and the other too large for anybody to really grasp, and the difference between the two is still drastically short of the difference we're talking about.

Answer 2

Your colleague is wrong, in this case result of operation < is enough. Comparing with epsilon could be necessary if you need to find out how many max elements in container etc. Just to find a max element simple comparison is enough.