Mocking parser object in a unit test

Question

I am trying to develop a unit test for the function that has a dependency on a parser (BeautifulSoup), which in turn depends on a network access to fetch a web page. In order to prevent network access I copied all HTML code to the file and whenever I need that web page I simply read it from a file. However, I have a hard time mocking the parser.

My question is: should I try to mock a parser, and if the answer is yes, then how?

Here is the method I am trying to test, which is inside data_processing.py :

def get_ocw_course_info(url):
    parser = get_parser(url)
    url_name  = parser.find('meta', {"name":"Search_Display"}).get('content').replace('|', '-')
    description = parser.find('meta', {"name":"Description"}).get('content')

    return dict(url=url,
                url_name=url_name,
                description=description)

And here is the unit test that I have developed for this function:

@patch('data_processing.get_parser')
    def test_get_ocw_course_info_unit(self, *args, **kwargs):
        data_processing.get_parser.return_value = BeautifulSoup(read_mock_html('mock_responses/ocw_pass.html'), 'lxml')

        actual = data_processing.get_ocw_course_info('https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-682-prototyping-avionics-spring-2006/assignments/')
        expected = {'url': 'https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-682-prototyping-avionics-spring-2006/assignments/', 
                    'url_name': '16.682 Prototyping Avionics - Assignments', 
                    'description': 'This section contains three of the four assignments from the class.', 
                    }
        self.assertEqual(actual, expected)

I omitted implementations of helper functions because they are either one liners or there is nothing particularly interesting going on (and I suppose names should be pretty self-explanatory)

Answer 1

I think you have two options here:

1. Easiest one, in my opinion, is to accept an optional parameter get_parser (which will be some kind of callable) in get_ocw_course_info procedure and use simple dependency injection to provide testable implementation to your function.

2. You can easily mock parser via mock.patch , and that's the way you doing it (don't quite understand what is your problem). Notice, in decorated test case you will get second argument, which is mock you can configure:

    @patch('data_processing.get_parser') def test_get_ocw_course_info_unit(self, mock, *args, **kwargs): mock.return_value = ... 

Or, the other way, you can provide predefined mock by passing it to patch decorator:

def get_parser_mocked(): # some suitable return value

@patch('data_processing.get_parser', get_parser_mocked)
def test_get_ocw_course_info_unit(self, *args, **kwargs):
    ...