How to Write Method to Verify Reflection Method Names and Parameters

Question

We were writing a Selenium test core by using Python webdriver. The main idea is to read from a CSV file with the format:

method_name,parameter 1,parameter 2, parameter 3, ..., parameter n

And then by using reflection, the test core will call the method based on the "method name" and parameters.

def runTest(self):
        try:
            test_case_reader = csv.reader(open(self.file_path, 'rb'), delimiter=',')
            self.logger.info("Running test case %s" % self.file_path)
            for row in test_case_reader:
                if (len(row) > 1):
                    method_name, parameters = row[0], row[1:]
                    parameters = filter(None, parameters)
                    method = getattr(self, method_name)
                    self.logger.info("executing method %s parameters %s" % (method_name, parameters))
                    method(*parameters)
                    self.wait()
        except AssertionError, e:
            self.logger.error(e)
            self.fail("Test case failed: %s" % self.file_path)

The main idea of this script is to provide a better productivity and ease of usage, since QA won't have to interact with python code to write automation test cases.

But due to human error, sometimes there will be a typo or parameter mismatched.

So I'm going to add a method to verify all CSV files if method names and parameters are correct. Is there any built-in reflection to do this kind of checking?

Update:

For the parameters I'd like to check if parameters count match the required parameters of the method. Some of the methods has optional parameters too.

Example:

method_1 (parameter1, parameter2, parameter3, parameter4 = None, parameter5 = None)

For method_1 , valid parameters count are: 3, 4, and 5.

Answer:

Not the perfect & tidy solution but this is the method for now and it's working:

def verifyFile(self):
    test_case_reader = csv.reader(open(self.file_path, 'rb'), delimiter=',')
    for row in test_case_reader:
        if (len(row) <= 1):
            break

        method_name, parameters = row[0], row[1:]
        if (method_name == ''):
            break

        parameters = filter(None, parameters)
        if not hasattr(self, method_name):
            self.logger.error("test case '%s' method name '%s': Method not found" % (self.file_path, method_name))
            break
        t = inspect.getargspec(getattr(self, method_name))
        max = len(t.args)
        if (type(t.defaults) == NoneType):
            min = max
        else:
            min = max - len(t.defaults)
        if not min <= len(parameters) + 1 <= max:
            self.logger.error("test case '%s' method name '%s': Parameter count not matches (%d - %d)" % (self.file_path, method_name, min, max))

Answer 1

For the method, you can easily call hasattr to see if it exists. In your case, it's just :

for row in test_case_reader:
    method_name = get_method_name() # Your code here
    if not hasattr(self, method_name):
        self.logger.error("No method name %s", method_name)

For the parameter, it's a little more complex, as python is a dynamic language. You may want to use a parameter checking or a contract library. If the parameter problem is simple enough, there may be a simpler solution. What do you want to check about parameters ?

EDIT :

For the parameter, you can use inspect.getargspec to get the number of arguments of the function and check if it's correct.

def check_args(func, n):
    args, varargs, keywords, defaults = inspect.getargspec(func)
    max = len(arg)
    min = max - len(defaults)
    if not min <= n <= max:
        # error

This don't take into account the use of *args and **kwargs but you can easily change it to do so.