With pytest, assert with "big" multiple line strings comparison are very slow

Question

With this test_strings.py:

def test_long_string_compare():
    t1 = 'this is a line that will be repeated many times \n' * 300
    t2 = '-> this is a line that will be repeated many times with some differences \n' * 300
    print('is equal :', t1==t2)
    assert t1 == t2

if __name__ == '__main__':
    test_long_string_compare()

The test is really long for comparing 300 lines strings:

========================================================================================================================== test session starts ==========================================================================================================================
platform linux -- Python 3.6.9, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /home/elapouya/tmp/tests
plugins: diff-0.1.14
collected 1 item                                                                                                                                                                                                                                                        

test_strings.py F                                                                                                                                                                                                                                                 [100%]

=============================================================================================================================== FAILURES ================================================================================================================================
_______________________________________________________________________________________________________________________ test_long_string_compare ________________________________________________________________________________________________________________________

    def test_long_string_compare():
        t1 = 'this is a line that will be repeated many times \n' * 300
        t2 = '-> this is a line that will be repeated many times with some differences \n' * 300
        print('is equal :', t1==t2)
>       assert t1 == t2
E       AssertionError: assert 
E         '-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated many times with some differences \n-> this is a line that will be repeated...
E         
E         ...Full output truncated (903 lines hidden), use '-vv' to show

test_strings.py:5: AssertionError
------------------------------------------------------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------------------------------------------------------
is equal : False
======================================================================================================================== short test summary info ========================================================================================================================
FAILED test_strings.py::test_long_string_compare - AssertionError: assert 
=========================================================================================================================== 1 failed in 9.61s ===========================================================================================================================

real    0m9,840s
user    0m9,817s
sys 0m0,020s

Almost 10s it is huge !

If I just execute the python script:

time python test_strings.py 
is equal : False
Traceback (most recent call last):
  File "test_strings.py", line 8, in <module>
    test_long_string_compare()
  File "test_strings.py", line 5, in test_long_string_compare
    assert t1 == t2
AssertionError

real    0m0,025s
user    0m0,020s
sys 0m0,004s

This is 400x faster

If use pytest --assert=plain , it is much faster:

time pytest -vv --assert=plain
========================================================================================================================== test session starts ==========================================================================================================================
platform linux -- Python 3.6.9, pytest-5.4.3, py-1.8.2, pluggy-0.13.1 -- /home/elapouya/.virtualenvs/tests-D190GBZr/bin/python3.6
cachedir: .pytest_cache
rootdir: /home/elapouya/tmp/tests
plugins: diff-0.1.14
collected 1 item                                                                                                                                                                                                                                                        

test_strings.py::test_long_string_compare FAILED                                                                                                                                                                                                                  [100%]

=============================================================================================================================== FAILURES ================================================================================================================================
_______________________________________________________________________________________________________________________ test_long_string_compare ________________________________________________________________________________________________________________________

    def test_long_string_compare():
        t1 = 'this is a line that will be repeated many times \n' * 300
        t2 = '-> this is a line that will be repeated many times with some differences \n' * 300
        print('is equal :', t1==t2)
>       assert t1 == t2
E       AssertionError

test_strings.py:5: AssertionError
------------------------------------------------------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------------------------------------------------------
is equal : False
======================================================================================================================== short test summary info ========================================================================================================================
FAILED test_strings.py::test_long_string_compare - AssertionError
=========================================================================================================================== 1 failed in 0.02s ===========================================================================================================================

real    0m0,200s
user    0m0,193s
sys 0m0,008s

But I am loosing string comparison.

I found a workaround:

def test_long_string_compare3():
    t1 = 'this is a line that will be repeated many times \n' * 300
    t2 = 'xx this is a line that will be repeated many times \n' * 300
    t1 = t1.splitlines()
    t2 = t2.splitlines()
    lt1 = len(t1)
    lt2 = len(t2)
    if lt1 > lt2:
        t2 += [''] * (lt1-lt2)
    if lt1 < lt2:
        t1 += [''] * (lt2-lt1)
    for i,(left,right) in enumerate(zip(t1,t2)):
        assert left == right, f'Line {i} differs'

But it stops on first difference.

Is there a way to get a full string comparison and not have such slowness?

Answer 1

I ran into this too. Ended up using testfixtures.compare , seems to work fine.