Automated integration tests as part of Azure DevOps CD/CI

Question

Basic Java CRUD application. Nothing crazy. We would want to test the data layer accordingly (so we have dependencies on a database).

The project is maven based, so the pipeline runs a build/runs unit tests then drops the artifact accordingly.

My question is this: what's the best practice to handle this with Azure Pipelines? Should we run an in-memory database that we can create and destroy once tests are complete (so basically keep the pipeline as is and just handle this as part of the maven tests that already run)? This would allow us the ability to control the data we start with and create an appropriate baseline for all tests.

Is it better to do that in the release pipeline after pushing to a docker container that we rebuild each time?

What is considered "best practice" for Azure DevOps (even DevOps in general)?

Answer 1

I think you should be running Mock objects in order to run tests and not use the real database. You can also encapsulate your tests in transactions and force them to fail and rollback the db after the tests are ran.

Answer 2

As the tests do not rely on a physical deployment to an environment I would bring the tests as far forward in your pipeline as possible, to the CI stage (Azure DevOps build stage), so meeting a DevOps principal to "fail fast".

This also has the advantage that the tests could easily also run during a pull request check.

There are trade-offs between running the integration tests against a real database or an in memory. The real database is likely to give you slightly more confidence, but the in memory database may result in a more reliable test suite.

Functional / acceptance / black box tests that run against a deployed application should certainly be placed in the release stages of your pipeline.

Good luck. Hope this helps.