How to unit-test the behavior of 'randomly' choosing one value?

Question

I have a function which is expected to randomly return one value from a collection of values. Is there a good way to test this random behavior with unit-test tools like JUnit etc.?

Answer 1

In situations that called for a lot of unit testing of code that normally behaves randomly, I've sometimes wrapped a stream of results from a java.util.Random in an Iterable<Integer> . The advantage is that, during unit testing, I can call the same method with an ArrayList<Integer> and get completely predictable behavior.

Answer 2

No. By definition, the result is (or should be) indeterminate, so the usual concept of "expected result" doesn't apply.

However, you could write a fairly simple test using a statistical approach, perhaps that when called n times, the set (unique) of values returned is at least .75 n, or something similar.

Another approach may be to defer the "randomness" to a trusted and sufficient implementation, such as an algorithm based on Math.random() , meaning your unit test would not have to test the randomness , rather just the functionality.

Answer 3

java.util.Random can probably be relied upon to be 'sufficiently random' for your purposes, so I assume what you're trying to test is that you get a proper random distribution among the items in your collection. It's true that there are a number of ways you could select an item out of a collection based on a random number and still end up biasing the results. For instance, iterating across the collection and using a random check at each stage will bias towards earlier items in the list.

If you want to test that your function actually produces random results you're going to need to find a statistical analysis toolkit which can do that. I'd suggest you fill up collections of various sizes with integer sequences, and then run tests of your random fetching code against those collections. You can feed the fetched values into the statistical analysis to determine if they're random or biased, and since they are linear sequences, the result should imply the same property for your fetching code as a whole.

Answer 4

The standard way of testing random values is to generate a few thousand of them, enumerate how many of each you get, calculate the chi-square statistic of the data set, then the incomplete gamma function will give you the probability of that distribution occurring at random. If that probability is too close to 0, it is likely that your RNG is biased.

The classic example of this is the "dieharder" test suite. You might also check out the test code in my http://github.com/lcrocker/ojrandlib.