Python: Run The Same Unittest module Tests For Multiple Files

Question

I am attempting to create a simple framework that will discover all of the test cases from a specific directory (I am using unittest for these cases) and run each of these test cases against multiples python files that will all implement the same code with the same function signatures.

Autograder.py  
TestCasesFolder/
    TestCase1.py  
    TestCase2.py  
    ...  
ImplementationFolder/
    Implementation1.py
SecondImplementationFolder/
    Implementation2.py

The framework succesfully finds all of the test case using (note this is in the class)

self.suites = unittest.defaultTestLoader.discover(self.testDirectory)

From there, I would like to run these suites on both Implementation1 and Implementation2.

I have been using the built in

self.suites.run(unittest.TestResult)

method from unittest to run my tests, and my first attempt at solving this problem was to import the current implementation I wanted to test using

imp.load_source

and then update the global namespace for the TestCase1.py with the correct module reference. However, because each module has its own global namespace I'm not sure if I can hook into the other files namespace. I am also not sure if this the correct approach, or if there is a better way than my implementation. How should I go about doing this?

EDIT My current solution that seems to work is for the Autograder.py file to update the __builtins__ module with a reference to the Implementation module. The actual line looks like:

__builtins__.ImplementationModule = imp.load_source("Implementation Module", "Implementation1.py")

This means when the TestCase1.py has access to ImplementationModule through __builtins__ . Of course the problem is this assumes that the __builtins__ module never implements anything that has the name ImplementationModule otherwise I will overwrite it with unknown implications. Is there a less risky version of doing this?

Answer 1

Have you looked at the nose system? It sounds very similar to what you are doing.

http://readthedocs.org/docs/nose/