Continuous Integration - Tests Categorization

Question

The current set up uses FinalBuilder for Continuous Integration & builds of a enterprise wide suite of ASP.NET applications & Services.

I intend to have NUnit hooked to the CI build process.

What should be my top priority category of tests - which if failed - should halt the process?

I plan to get with the Product Owners to find out the top functionalities that they cannot do without. But from a generic perspective what would/should be considered critical enough to halt the process?

Answer 1

I like to run the newest tests first when possible, followed by the most commonly failed tests, and finally all remaining tests.

Your tests need to help developers/QA pinpoint issues; they aren't there exclusively to make management happy. The product owners' favourite tests will still run, but why make the rest of the build wait while those tests are executed? (Management would have everything as priority #1 anyways)

Any build-killing test will be worth a red flag. The faster you can fail any of these tests, the more productive your team will be as a whole.