How to enforce relative constraints on hypothesis strategies?

Question

Say I have 2 variables a and b where it is given that b > a , how then can I enforce this relative constraint on the hypothesis strategies?

from hypothesis import given, strategies as st

@given(st.integers(), st.integers())
def test_subtraction(a, b):
  # only evaluates to true if b > a
  # hence I'd like to enforce this constraint on the strategy
  assert abs(b - a) == -(a - b)

Answer 1

(I've no idea how to do this in Python, but it's a common enough problem, so I hope you can use these F# FsCheck examples instead. The ideas are universal.)

Filtering

Most property-based frameworks come with an ability to filter values based on a predicate. In FsCheck it's the ==> operator. In QuickCheck the equivalent is called suchThat .

Using the ==> operator in FsCheck, you can write the property like this:

[<Property>]
let property_using_filtering (a : int) (b : int) =
    b > a ==> lazy
    Assert.Equal (abs (b - a), -(a - b))

(It's possible to write the test in a more terse and idiomatic style, but since I'm assuming that you may not be familiar with F#, I chose to be more explicit than usual.)

Notice that the predicate b > a precedes the filtering operator ==> . This means that the rest of the code to the right of, and below, the operator only runs when the predicate is true.

The framework is still going to generate entirely random values, so (assuming a uniform random distribution) it'll be throwing half of the generated values away.

Thus, to generate 100 (the default) valid test cases, it'll have to generate on average 200 test cases (ie 400 integers). Generating 400 integers instead of 200 integers probably isn't a big deal, but in general, this kind of filtering can be prohibitively wasteful.

Therefore, it's always useful to be aware of alternatives.

Seed and diff

When faced with this sort of problem, it usually helps to take an alternative look at how to generate values. How do you generate two values where one is strictly greater than the other?

You can generate a random value (the seed), which in this case will also serve as the first value itself. Then a second value will indicate the difference between the two.

Some property-based frameworks come with features where you can tell it to generate strictly positive numbers. FsCheck comes with those features, but assuming for the moment that not all frameworks can do this, you can still use an unconstrained random value.

In that case, the difference, being any random number, may be both negative, zero, or positive. In this case, we can take the absolute value of the number and then add one to ensure that it's strictly greater than zero. Now you have a number that's guaranteed to be greater than zero. If you add that to the first number, you're guaranteed to have a number greater than the first one:

[<Property>]
let property_using_seed_and_diff (seed : int) (diff : int) =
    let a = seed
    let b = a + 1 + abs diff
    Assert.Equal (abs (b - a), -(a - b))

Here, (somewhat redundantly) we set a = seed and then b = a + 1 + abs diff according to the above description.

(I only included the redundant seed function parameter to illustrate the general idea. Sometimes, you need one or more values calculated from a seed, but not the seed itself. In the present case, however, the value and seed coincide.)

Answer 2

You can add these constrains using assume in hypothesis:

from hypothesis import given, strategies as st, assume

@given(st.integers(), st.integers())
def test_subtraction(a, b):
  assume(b > a)
  # only evaluates to true if b > a
  # hence I'd like to enforce this constraint on the strategy
  assert abs(b - a) == -(a - b)

See: https://hypothesis.readthedocs.io/en/latest/details.html#making-assumptions for more details

Answer 3

In addition to the filtering and seed-plus-diff approaches shown above, the "fix-it-up" approach can be useful: try to generate something valid, and just patch the object if it's not satisfied. In this case, that might look like:

@given(st.integers(), st.integers())
def test_subtraction(a, b):
    a, b = sorted([a, b])
    ....

The advantage here is that the minimal failing example tends to look a bit more natural, and might have a nicer distribution than a seed-and-diff ( or "constructive" ) approach. It also combines well with the other approaches, especially if you're defining your own strategy with @st.composite .