Harvesting function outputs (as well as inputs) from Haskell Quickcheck

Question

What is the best way to tuple: 1. QuickCheck inputs to a predicate with 2. the outputs returned by a tested function ?

Wanting to test (2 * 350) functions in a couple of other languages (to check for divergence from the behaviour of the Haskell base and Prelude functions which they imitate) my first experiment has been to:

• capture output from (QuickCheck) verboseCheck result strings
• apply the Haskell function to each of the (re-read) inputs which QuickCheck threw at it, to obtain the corresponding Haskell output
• use these inputs and outputs in automated checks of the parallel functions in other languages for divergences from the behaviour of the Haskell function

For example, to get test data for a function on a list, something like:

testSet1
  :: (Show a, Testable prop)
  => String -> String -> ([Int] -> a) -> prop -> IO ()
testSet1 folder fname f p = do
  r <- verboseCheckResult p
  let ys =
        ((,) <*> f) . read . listTuple1 <$>
        splitOn ["Passed:"] (tail $ init $ (lines . output) r)
  writeFile (concat [folder, fname, ".txt"]) (show ys)

This works OK, at least for simple cases, but I cannot immediately find ways of generalising this kind of pattern to avoid having to write a separate function (to generate and capture inputs with outputs) for every variant of type signature.

The key problem is that read typically needs some type information with which to parse the list of QuickCheck-generated inputs from their (QuickCheck result output) stringifications.

Perhaps I am taking the wrong approach. Any thoughts ? Template Haskell ?

Answer 1

There is a separate package to annotate properties with counterexamples (which can consist of inputs and outputs): quickcheck-with-counterexamples .

This QuickCheck issue on Github has some discussion on the same topic.