3-way comparison of floating-point numbers

Question

Given a pair of floating-point numbers, what's the best way to perform a three-way comparison, that is, return negative, zero or positive, depending on whether the first number is less than, equal to or greater than the second number?

Some languages like Perl, Java and C++20, have this operator built-in, or in the standard library. However, I'm asking how to do this in plain C.

If the inputs were integers, it would be a simple matter of writing a pair of two-way comparisons. But it's more complicated with floating-point inputs because of the special behavior of NaNs in comparison. What's the appropriate way to do it, taking that into account?

Answer 1

Returns -1, 0, 1 for less, equal and greater, plus some other value for not comparable:

#include <math.h>

int cmp(double a, double b) {
    if (isunordered(a, b)) {
        /* at least one NaN, return whatever you feel is appropriate */
        return 42;
    }
    return (a>b) - (a<b);
}

If you want fuzzy comparisons with epsilon, feel free to make the < and > bit more complicated...

Answer 2

What's the appropriate way to do it, taking that (NAN) into account?

Return a FP type to allow 4 different returns values: -1.0, 0.0, 1.0, NAN.

Below also returns -0.0 in select cases involving -0.0.

#include <math.h>

double fcmp(double a, double b) {
  if (isunordered(a, b)) return NAN;
  if (a > b) return 1.0; 
  if (a < b) return -1.0; 
  return a - b;
} 

---

I'd even consider propagating the a or b when one is NAN to maintain the NAN payload. There may exist many different non-a-numbers.

double fcmp(double a, double b) {
  if (isnan(a)) return a;
  if (isnan(b)) return b;
  ...
} 

---

But let us look at a 3-way used for sorting as with qsort() . A question is where to put NANs? A common goal is to put them at the end is the list - that is all NAN are greater than others. To do so, we need to consistently compare, even if both operands are NANs with different payloads.

// All NAN considered greater than others
// return 0, a positive or negative int.
int fcmp_for_qsort(const void *ap, const void *bp) {
  double a = *(const double *) ap;
  double b = *(const double *) bp;
  if (isnan(a)) {
    if (isnan(b)) {
      return 0;
    }
    return 1;
  }
  if (isnan(b)) -1;
  return (a > b) - (a < b);
}