Subclassing TestCase in Python: Overwriting Field in Parent TestCase

Question

I'm writing integration tests for an Alexa app.

Our application uses a controller-request-response pattern. The controller receives a request with a specified intent and session variables, routes the request to functions that do some computation with the session variables, and returns a response object with the results of that computation.

We get the right behavior from UnhandledIntentTestCase as far as test_for_smoke is concerned. However, test_returning_reprompt_text never fires, because returns_reprompt_text is never overwritten.

Can someone explain how I can overwrite it in the parent class and/or how the correct intent name is passed to the request object in setUpClass?

intent_base_case.py

import unittest

import mycity.intents.intent_constants as intent_constants
import mycity.mycity_controller as mcc
import mycity.mycity_request_data_model as req
import mycity.test.test_constants as test_constants




###############################################################################                                                                
# TestCase parent class for all intent TestCases, which are integration tests #                                                                
# to see if any changes in codebase have broken response-request model.       #                                                                
#                                                                             #                                                                
# NOTE: Assumes that address has already been set.                            #                                                                
###############################################################################                                                                

class IntentBaseCase(unittest.TestCase):

    __test__ = False

    intent_to_test = None
    returns_reprompt_text = False

    @classmethod
    def setUpClass(cls):
        cls.controller = mcc.MyCityController()
        cls.request = req.MyCityRequestDataModel()
        key = intent_constants.CURRENT_ADDRESS_KEY
        cls.request._session_attributes[key] = "46 Everdean St"
        cls.request.intent_name = cls.intent_to_test
        cls.response = cls.controller.on_intent(cls.request)

    @classmethod
    def tearDownClass(cls):
        cls.controller = None
        cls.request = None

    def test_for_smoke(self):
        self.assertNotIn("Uh oh", self.response.output_speech)
        self.assertNotIn("Error", self.response.output_speech)

    def test_correct_intent_card_title(self):
        self.assertEqual(self.intent_to_test, self.response.card_title)


    @unittest.skipIf(not returns_reprompt_text,
                     "{} shouldn't return a reprompt text".format(intent_to_test))
    def test_returning_reprompt_text(self):
        self.assertIsNotNone(self.response.reprompt_text)


    @unittest.skipIf(returns_reprompt_text,
                   "{} should return a reprompt text".format(intent_to_test))
    def test_returning_no_reprompt_text(self):
        self.assertIsNone(self.response.reprompt_text)

test_unhandled_intent.py

import mycity.test.intent_base_case as base_case


########################################                                                                                                       
# TestCase class for unhandled intents #                                                                                                       
########################################                                                                                                       


class UnhandledIntentTestCase(base_case.IntentBaseCase):

    __test__ = True

    intent_to_test = "UnhandledIntent"
    returns_reprompt_text = True

output

======================================================================
FAIL: test_correct_intent_card_title (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 44, in test_correct_intent_card_title
    self.assertEqual(self.intent_to_test, self.response.card_title)
AssertionError: 'UnhandledIntent' != 'Unhandled intent'
- UnhandledIntent
?          ^
+ Unhandled intent
?          ^^


======================================================================
FAIL: test_returning_no_reprompt_text (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 56, in test_returning_no_reprompt_text
    self.assertIsNone(self.response.reprompt_text)
AssertionError: 'So, what can I help you with today?' is not None

----------------------------------------------------------------------

Answer 1

This is because of execution order. The SkipIf decorators are executed once during the parsing of the IntentBaseCase class. They aren't re-executed for each class or for each call to the test function.

The decorator pattern for SkipIf is designed for use with fixed global variables such as versions of dependent modules, operating system or some other external resource who's availability can be calculated or known in the global context.

Skipping tests is also something that should be done for external reasons, not for internal ones such as the needs of a sub-class. A skip is still a kind of failing test which is indicated in the report so you can see your test suite isn't exercising the whole of the functional scope of the project.

You should redesign your base class structure so functions are only available to run if the sub-class and skip using Skip for this. My recommendation would be:

class IntentBaseCase(unittest.TestCase):
    ...

class RepromptBaseCase(IntentBaseCase):
    def test_returning_reprompt_text(self):
        self.assertIsNotNone(self.response.reprompt_text)

class NoRepromptBaseCase(IntentBaseCase):
    def test_returning_no_reprompt_text(self):
        self.assertIsNone(self.response.reprompt_text)

You should also consider moving the response portion out of the setUp and put it into a test_ function of it's own and change these test_returning functions into a simpler assertReprompt and assertNoReprompt functions. It's a good idea to set up the tests in setUp, but not a good idea to run the actual code there.