python hypothesis unit test using faker

Question

I have a problem with python unit test by using hypothesis additional package, Faker . I want to test login process of my website, I already have the unit test scenario, but I want to automated the scenario with hypothesis . This is my simple code for email and password validation.

import unittest

from Environtments.ParametrizedTestCase import ParametrizedTestCase
from hypothesis import given
from faker import Faker

class TestLogin(ParametrizedTestCase):
    fake = Faker('en_US')
    @given(email = fake.email(),password = fake.password()
    def test_login(self,email,password):
        assert '@' in email
        assert len(password) >= 6

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTest(ParametrizedTestCase.parametrize(TestLogin, param=""))
    unittest.TextTestRunner(verbosity=2).run(suite)

I always find this error. Is there something I missed?

test_login (__main__.TestLogin) ... You can add @seed(311936867547523412638507752560457398354) to this test to reproduce this failure.
ERROR

======================================================================
ERROR: test_login (__main__.TestLogin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tester_uf_exmp_hypo.py", line 10, in test_login
    def test_login(self,email,password):
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 1001, in wrapped_test
    state.run()
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 725, in run
    runner.run()
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 435, in run
    self._run()
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 771, in _run
    self.reuse_existing_examples()
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 626, in reuse_existing_examples
    self.test_function(last_data)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 153, in test_function
    self._test_function(data)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 691, in evaluate_test_data
    escalate_hypothesis_internal_error()
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 663, in evaluate_test_data
    result = self.execute(data, collect=True)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 578, in execute
    result = self.test_runner(data, run)
  File "/Library/Python/2.7/site-packages/hypothesis/executors.py", line 58, in default_new_style_executor
    return function(data)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 552, in run
    args, kwargs = data.draw(self.search_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/data.py", line 158, in draw
    return self.__draw(strategy, label=label)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/data.py", line 175, in __draw
    return strategy.do_draw(self)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 180, in do_draw
    return self.base.do_draw(data)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/lazy.py", line 157, in do_draw
    return data.draw(self.wrapped_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/data.py", line 148, in draw
    if strategy.is_empty:
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 144, in accept
    recur(self)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 141, in recur
    mapping[strat] = getattr(strat, calculation)(recur)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/collections.py", line 67, in calc_is_empty
    return any(recur(e) for e in self.element_strategies)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/collections.py", line 67, in <genexpr>
    return any(recur(e) for e in self.element_strategies)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 141, in recur
    mapping[strat] = getattr(strat, calculation)(recur)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 541, in calc_is_empty
    return recur(self.mapped_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 141, in recur
    mapping[strat] = getattr(strat, calculation)(recur)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/lazy.py", line 87, in calc_is_empty
    return recur(self.wrapped_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/lazy.py", line 110, in wrapped_strategy
    *self.__args, **self.__kwargs
  File "/Library/Python/2.7/site-packages/hypothesis/strategies.py", line 651, in fixed_dictionaries
    check_strategy(v)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/validation.py", line 45, in check_strategy
    check_type(SearchStrategy, arg, name)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/validation.py", line 39, in check_type
    % (typ_string, name, arg, type(arg).__name__))
InvalidArgument: Expected SearchStrategy but got u'2)Y4+APqu@' (type=unicode)

----------------------------------------------------------------------
Ran 1 test in 2.013s

FAILED (errors=1)

I change the code to this.

from hypothesis.extra.fakefactory import fake_factory    
@given(email = fake_factory('email'),password = fake_factory('password'))

This is the warn. But the faker works. Is this will be a problem?

tester_uf_exmp_hypo.py:9: HypothesisDeprecationWarning: hypothesis.extra.fakefactory has been deprecated, because it does not support example discovery or shrinking.  Consider using a lower-level strategy (such as st.text()) instead.
  @given(email = fake_factory('email'),password = fake_factory('password'))
test_login (__main__.TestLogin) ... kaareberge@pedersen.no
!l3QCGDd8I
myslrshd@bn.com
(@s(RwUM0z

Answer 1

Your first case is calling @given with arguments of the wrong type - Hypothesis requires them to be "strategies" for generating example values, rather than an value itself. You can see this at the bottom of the traceback - got u'2)Y4+APqu@' (type=unicode) is the result of fake.password() , rather than (eg) hypothesis.strategies.text() .

The second code snippet does work - hurray!

However - and here's why I wrote that deprecation warning - it's giving up most of the advantages of Hypothesis: because this is just a wrapper around fake-factory, examples can't be minimised and the testing process doesn't have enough information to do coverage-guided optimisation.

Instead, you can think about what a password is - in this case, perhaps "a unicode string with between 6 and 40 characters". In Hypothesis, that would be:

import hypothesis.strategies as st

@given(password=st.text(min_size=6, max_size=40))
def test(password): 
    ...

And then you get shrinking, profile-guided example selection, and a whole lot more :D

"Email addresses" are also strings, but with more structure - luckily, Hypothesis has that covered too. You can compose strategies from parts with .map() , or use the from_regex() strategy, for example, or just copy our internal implementation from hypothesis.provisional.emails (or even just import it, if you don't mind unstable APIs that could be removed at any time).

TLDR: you can mix Hypothesis and fake-factory, but I'd just use Hypothesis.