How to sort python unittest discover tests?

Question

I created my bunch of Python tests using the Python unittest format.

Now I'm able to run them with

python -m unittest discover -s TestDirectory -p '*.py' -v

I finds them all, and runs them.

Now there's a subtle difference whether I run the tests on Windows, or on Linux. Indeed, on Windows the tests are run in alphabetical order, whereas on Linux the tests are run in no apparent human specific discoverable order, even if always the same.

The trouble is I relied on the first two letters of the test file to sort the order of execution of the tests. Not that they have to be run in a specific order, but to have some kind of informational tests, showing version data in their output to appear first in the test run log.

Is there something I can do to run the tests also in alphabetical order on Linux?

Answer 1

I have not tried this, but I imagine that one thing you could do is override the TestSuite class to add a sort function. Then you can call the sort function before calling the unittest run function. So, in an 'AllTests.py' script, you could add something like this:

class SortableSuite(unittest.TestSuite):
    def sort(self):
        self._tests.sort(cmp=lambda x,y: x._testMethodName < y._testMethodName)
    def run(self,testResult):
        #or if you don't want to run a sort() function, you can override the run
        #function to automatically sort.
        self._tests.sort(cmp=lambda x,y: x._testMethodName < y._testMethodName)
        return unittest.TestSuite.run(self,testResult)

loader = unittest.TestLoader()
loader.suiteClass = SortableSuite
suite = loader.loadTestFromTestCases(collectedTests)
suite.sort()
suite.run(defaultTestResult())