Test.QuickCheck not generating random values correctly

Question

I am using Test.QuickCheck to generate random Sudoku puzzles.

data Sudoku = Sudoku { getSudoku :: [[Maybe Int]] } deriving (Show, Eq)

rows :: Sudoku -> [[Maybe Int]]
rows (Sudoku rs) = rs

--B1
printSudoku :: Sudoku -> IO ()
printSudoku s = do
    putStrLn . unlines . map (map (maybe '.' (head . show))) $ rows s

--C1
cell :: Gen (Maybe Int)
cell = suchThatMaybe (frequency [(90, choose (0,0)),(10, choose(1,9))]) (/=0)

--C2
instance Arbitrary Sudoku where
    arbitrary = do
        rows <- sequence [ sequence [ cell | j <- [1..9] ] | i <- [1..9] ]
        return (Sudoku rows)

--C3
prop_Sudoku :: Sudoku -> Bool
prop_Sudoku = isSudoku

checkRandomSudoku :: IO [Bool]
checkRandomSudoku = do
    s <- sample' (arbitrary :: Gen Sudoku)
    return $ map isSudoku s

The code runs fine. However, when I perform

a <- sample' (arbitrary :: Gen Sudoku)

sequence $ map printSudoku a

it returns something like this :

....5..3.
...4.....
...2.....
...5.....
.........
...33....
...5.....
...2.4...
.........

.........
.343.....
.........
......9.2
.........
45....5.1
.2......7
.7..88.34
.9....6..

....2..8.
.2121638.
.7.7...9.
4..45.6..
.....6.2.
..6.6....
53..9.6..
..9....7.
.47892...

.373411..
5...3282.
...45..9.
8989..18.
31.8113..
9..35.6..
4.685....
.4....39.
7..6.5.76

48.178.53
1.871.4.4
3165.17..
.1...7.59
.98126.51
6.6...775
9.4636952
.5..239..
372.....8

.34.73129
.5.8.27.1
344.34931
28.6.94.1
6327.3..8
3743.5496
93...7984
..82.8...
..3.54.93

273847853
5568.7465
832.73515
3766..6.7
.7.196256
1.96.9.3.
.7156.268
1615.196.
.392..633

731652284
863.8.768
31..5.5.6
961.5.467
1245.1159
5..275471
52.727759
6.656.849
99.72352.

which obviously is not random at all. The distribution of empty cells is very high initially and then slowly decreases. Am I using the wrong function or in the wrong way? Thanks

Answer 1

The reason you are seeing this behaviour is that QuickCheck tries progressively bigger test cases. This is described in the resizing section of the manual.

In the case of suchThatMaybe increasing the size makes it retry generating an arbitrary to match the predicate you passed to it. You can see that in the source http://hackage.haskell.org/package/QuickCheck-2.6/docs/src/Test-QuickCheck-Gen.html#suchThatMaybe . The interesting code is:

try _ 0 = return Nothing
try k n = do x <- resize (2*k+n) gen
             if p x then return (Just x) else try (k+1) (n-1)

What's happening is that suchThatMaybe retries n times where n depends on the size parameter passed to the generator.

sample' tries the sizes [0,2..20] which get propagated all the way to suchThatMaybe .

You can override the increasing sizes by calling resize on a generator:

>>> a <- sample' $ (resize 2 arbitrary :: Gen Sudoku)
>>> sequence $ map printSudoku a
.....8.5.
.......4.
.........
4..25...5
......9..
....5....
........7
.....7...
...4.....

...5.....
.......4.
...6...6.
..14.....
...7...7.
....2....
.....6...
...4..572
.4.....6.

..6..8...
..4......
.........
......9..
839......
67..4....
.5.......
....5....
........3

4...1....
7..9.39..
....6....
...4.1...
.........
.........
..9....6.
.2.9...84
.....8...

.64......
..3.44...
4.......4
....1..8.
.9.......
34.......
.....6...
18.2..593
.4.7.....

.........
...8..6..
.2......5
...5.....
.2.......
........6
.3....13.
8.1.2..85
....5....

..7......
..67...5.
..6......
27....1.9
.9.......
78.....7.
......34.
.......2.
..81...81

3.1......
.........
.....6...
.........
.16.71...
.........
.2.......
.........
.....9.1.

..65..6.9
........5
..1.4....
....86...
.2..2..2.
.....9...
..6......
.........
...7..855

.......94
...14..8.
.....4...
...3....9
.........
.....5...
.5.......
45.....8.
..48.....

4........
......3..
5......4.
.4..6..2.
..3......
.........
..9......
6..9.....
....7....