What is the fastest way of testing correctness of a sorting function?

Question

Using a genetic algorithm, I found this comparison list:

compareAndSwap(x[0],x[2]);
compareAndSwap(x[3],x[4]);
compareAndSwap(x[2],x[4]);
compareAndSwap(x[0],x[3]);
compareAndSwap(x[2],x[3]);
compareAndSwap(x[1],x[3]);
compareAndSwap(x[1],x[2]);
compareAndSwap(x[0],x[1]);
compareAndSwap(x[3],x[4]);

but I need to test it if it works for all cases. Also number of array elements(currently 5) can grow up to 100 in some situations. This would mean that number of cases to check against is growing fast like more than pow(2,100) .

If I give an oppositely sorted array alone as a worst case, that doesn't check against any error about middle element x[2] comparisons. For example, 5,4,3,2,1 is sorted by some function into 1,2,3,4,5, by

compareAndSwap(x[0],x[4]);
compareAndSwap(x[1],x[3]);

alone and this certainly doesn't sort many cases of 5-element arrays.

Tried random number generators for sample arrays but not sure if its acceptable:

      std::random_device rd;
      std::mt19937 rng(rd());
      std::uniform_real_distribution<double> dist(0,1);

      for(int k=0;k<500;k++)
      {
        std::vector<double> arraySorted;
        for(int i=0;i<5;i++)
            arraySorted.push_back(dist(rng));

      //sortNetwork(arraySorted.data());

      //if(!std::is_sorted(arraySorted.begin(),arraySorted.end())) 
            throw std::runtime_error("error");
      }

even this can still miss some parts. Is there a fast way to test sorting algorithms?

What if it was 1000 elements array? Are these tested using math, pen and paper within some theorems and known algorithms or using supercomputers?

Just some sample cases for 4 elements:

1 2 3 4   
1 2 4 3   
2 1 3 4    
2 1 4 3   
1 2 0 1                        
1 2 1 0                         
2 1 0 1
2 1 1 0
3 4 2 1                           
3 4 1 2
4 3 2 1
4 3 1 2
1 1 1 1

seems to have more than pow(2,n) cases.

Can a sorting network be treated like a graph problem when generating test data, somehow?

Answer 1

While you could check every single iteration of every single possible list, as you've pointed out this would be far too slow. Testing is not about proving the algorithm correct , for that you'd need to do a proof. Testing is about reducing the possibility of a bug by testing all the places it might hide. Testing rarely attempts to cover the entire possible space, but rather possible types of errors.

Here's some examples to exercise a sort function.

• An empty list
• A single element list
• A list with all zeros
• An ordered list
• A reversed list
• A list of all the same elements
• A very large list
• A list with strange elements (for example, Unicode, negative numbers, overloaded numbers)

Then there's erroneous inputs which should return an error rather than garbage. Garbage in, error out.

• A null pointer
• A list of nulls
• A list which is too large (if your function has size limits)

And yes, randomize. Generate random valid lists of random valid sizes and then verify the result of the sort is in order. This helps cover any cases you might have missed and avoids any bad assumptions you may have made. This is particularly important when testing a function "black box" meaning the tester has no knowledge of its internals. Every time you run more random lists against the function you further reduce the possibility there is a bug.

Be sure to output the random seed used so you can repeat the test if there is a failure.

Finally, use test coverage to ensure your tests are hitting all lines and branches of the code. The code might be generated by an AI, but you can still do a coverage analysis on it to identify your testing gaps. Running a code beautifier over the probably unreadable AI generated code will help your understanding of where your need more tests.