Haskell QuickCheck best practices (especially when testing type classes)

Question

I've just started using QuickCheck with a bunch of Haskell code. I'm behind the times, I know. This question is a two-parter:

Firstly, what are the general best-practices for Quick Check? So far, I've picked up the following:

• Name your tests prop_* (annoying, because everything else is camelCase)
• Test exported code (if you're testing internals you're likely doing it wrong)
• Test properties, not examples
  • Don't say X is out of range, Y is in range
  • Instead, say if x is out of range, normalize x ≠ x (or some other such property)

But I'm still grasping at other best practices. Particularly:

• Where are properties kept?
  • The same file?
  • in a test/ directory? (If so, then how do you import the stuff in src/ ?)
  • in a Properties/ directory under src ?

Most importantly, how do we tend to go about testing properties on type classes? For example, consider the following (simplified) type class:

class Gen a where
    next :: a -> a
    prev :: a -> a

I'd like to test the property ∀ x: prev (next x) == x . Of course, this involves writing tests for each instance. It's tedious to write the same property for each instance, especially when the test is more complicated. What's the standard way to generalize such tests?

Answer 1

It's tedious to write the same property for each instance

You don't do this. You write the property once for the class:

class Gen a where
    next :: a -> a
    prev :: a -> a

np_prop :: (Eq a, Gen a) => a -> Bool
np_prop a = prev (next a) == a

Then to test it, you cast to a particular type:

quickCheck (np_prop :: Int -> Bool)
quickCheck (np_prop :: String -> Bool)

Your other questions I can't help with.

Answer 2

I believe the prop_ convention came from QC coming with a script that ran all functions that started with prop_ as tests. So there's no real reason to do so, but it does visually stand out (so the property for a function foo is prop_foo ).

And there's nothing wrong with testing internals. There are two ways of doing so:

• Put the properties in the same module as the internals. This makes the module bigger, and requires an unconditional dependency on QC for the project (unless you use CPP hackery).

• Have internals in a non-exported module, with the functions to actually be exported re-exported from another module. Then you can import the internal module into one that defines the QC properties, and that module is only built (and has a QC dependency) if a flag specified in the.cabal file is used.

If your project is large, then having separate src/ and test/ directories may be useful (though having a distinction may prevent you from testing internals). But if your project isn't all that big (and resides under one overall module hierarchy anyway), then there's no real need to split it up like that.

As Norman Ramsey said in his answer, for type classes you can just define the property as being on the typeclass and use accordingly.

Answer 3

Try

{-# LANGUAGE GADTs, ScopedTypeVariables #-}
import Test.QuickCheck hiding (Gen)

class Gen a where
  next :: a -> a
  prev :: a -> a

np_prop :: SomeGen -> Bool
np_prop (SomeGen a) = prev (next a) == a

main :: IO ()
main = quickCheck np_prop

instance Gen Bool where
  next True = False
  next False = True
  prev True = False
  prev False = True

instance Gen Int where
  next = (+ 1)
  prev = subtract 1

data SomeGen where
  SomeGen :: (Show a, Eq a, Arbitrary a, Gen a) => a -> SomeGen

instance Show SomeGen where
  showsPrec p (SomeGen a) = showsPrec p a
  show (SomeGen a) = show a

instance Arbitrary SomeGen where
  arbitrary = do
    GenDict (Proxy :: Proxy a) <- arbitrary
    a :: a <- arbitrary
    return $ SomeGen a
  shrink (SomeGen a) =
    map SomeGen $ shrink a

data GenDict where
  GenDict :: (Show a, Eq a, Arbitrary a, Gen a) => Proxy a -> GenDict

instance Arbitrary GenDict where
  arbitrary =
    elements
    [ GenDict (Proxy :: Proxy Bool)
    , GenDict (Proxy :: Proxy Int)
    ]

data Proxy a = Proxy

The type class is reified into an existentially quantified dictionary, on which an Arbitrary instance is defined. This Arbitrary dictionary instance is then used to define an instance of Arbitrary for existentially quantified values.

Another example is given at https://github.com/sonyandy/var/blob/4e0b12c390eb503616d53281b0fd66c0e1d0594d/tests/properties.hs#L217 .

This can be further generalized (and the boilerplate reduced) if you are willing to use ConstraintKinds . The following is defined only once.

data Some c where
  Some :: (Show a, Arbitrary a, c a) => a -> Some c

instance Show (Some c) where
  showsPrec p (Some a) = showsPrec p a
  show (Some a) = show a

instance Arbitrary (Dict c) => Arbitrary (Some c) where
  arbitrary = do
    Dict (Proxy :: Proxy a) :: Dict c <- arbitrary
    a :: a <- arbitrary
    return $ Some a
  shrink (Some a) =
    map Some $ shrink a

data Dict c where
  Dict :: (Show a, Arbitrary a, c a) => Proxy a -> Dict c

data Proxy a = Proxy

class (c a, d a) => (c &&# d) a
instance (c a, d a) => (c &&# d) a

For each type class you want to test, an Arbitrary instance of Dict is required.

instance Arbitrary (Dict (Eq &&# Gen)) where
  arbitrary =
    elements
    [ Dict (Proxy :: Proxy Bool)
    , Dict (Proxy :: Proxy Int)
    ]

np_prop :: Some (Eq &&# Gen) -> Bool
np_prop (Some a) = prev (next a) == a