同时运行unittest测试用例和机器人框架测试用例

Question

Our group is evaluating Robot Test Framework for our QA group, not just for BDD, but also to possibly cover a lot of our regular functionality testing needs. It certainly is a compelling project.

To what extent, if any, is Robot Framework based on xunit (unittest) architecture? I see that unittest asserts can be used, but I don't see that the RF testcases themselves are based on unittest.TestCase.

Ideally, our organization would like to be able to be able to write Robot Framework tests, as well as Python unittest testcases, run the testcases together from one runner and get integrated results, reuse RF's Selenium2 Library's "keywords" as functions used by our regular unittest testcases in order to share a common SE code-base.

Is this a solved problem? Does anybody do this kind of thing?

Answer 1

RobotFramework is not the right tool for unit testing.

• Unit-tests should be written in the same language of the units (modules, classes, etc.)
• The ability to describe scenarios in natural language (which is one of the strongest features of systems like RF) is worthless in unit tests. At this level of testing scenarios are for input x you get output y .

RF is best suited in Acceptance Testing and Integration Testing, the top-grained verification of your system.

Nevertheless you can integrate RF and xunit in your QA system together. And merge reports from RF and unit-test.

Answer 2

While the goals of code unittesting and functional/non-functional testing are worlds apart, the python unittest module can lend itself as a framework for QA testing and unittesting. Robot framework was designed from the ground up for QA testing with hooks into languages such as Python/Java.

Robot framework works well for tabular test cases with pretty HTML output. While unittest does quite well with what it name implies, it can also lend itself well to non-unit tests with asserts calling functional test code directly. There are plugins for unittest output too such as HTMLTestRunner that have a similar 'pretty' nature of Robot framework html output.

We used Robot framework heavily for our QA testing at work, and we decided to switch to all of the test cases to the unittest module. While I understand it is not the intent of the module to be a functional test bed, Robot framework was proving to be a layer of complication from the test cases we needed to write and execute. This was causing more time and effort hooking in the native test code into Robot, with tougher and tougher debugging as the test suite grew. With unittest, we execute the same tests in native python (Robot's built-in features and plugins were too vanilla for the complexity of our project). We now rely on the python language directly with full IDE support during QA test development, then report back with whichever CI tool/reporting mechanism.

Answer 3

Robot is not at all based on xunit technologies. Personally I think it makes a great unit testing framework for python code, since you can create keywords that can directly import your modules. I use this technique for some projects I work on.

With robot, you can tag your unit tests or put them all in a separate hierarchy so that you can run them separate from acceptance tests if you like, or combine them and get statistics broken out separately.