What is the best way to run integration tests using a web server?

Question

As the title says, I'm interested in the best approaches of testing an application using a Jetty web server, taking in consideration that you don't want to start/stop the test server for each particular test.

As far as I know, these are the solutions:

1. if you use a build tool like Maven or Gradle, you can do it in the *.pom or *.gradle file.

   • this has the disadvantage that you have to create the test server in a different way that you normally do it in the application.

2. Creating a test suite and using the @BeforeClass and @AfterClass annotations to start/stop the server before/after the test Suite .

   • disadvantage here is the "ugly" way of specifying the tests to run. You also have to specify that the test which is added to the suite, it shouldn't run outside of the suite(avoid duplicate run of the test). I think Junit is not yet fully equipped for this.

3. Create and start the server in a static way at the beginning of the tests and use the ShutDown mechanism to hook up in the JVM and stop the server automatically when all the tests are completed. This seems to be the best solution because this mechanism is available in Jetty already, but

   • the disadvantage is that you are not in control of the stopping of the server. It is done actually in a totally different thread, even outside of the building tool(I use Gradle for this)

Answer 1

We do #2 for pretty much all of our tests in jetty itself. It is really very easy to create an embedded server and deploy an app, or a servlet, or whatever you like in it. This gives you more then just the ability to run automated tests but also a way to debug your application simply within an IDE, without all the tooling overhead of an ide. So from the Jetty perspective we are big fans of creating your test suite and starting and stopping servers as needed within that test. We have test cases where we spin up multiple servers and test session expiration between them, others where we spin up a server and toss 10k client connections against it with our async jetty-client. Just running a complete build of jetty starts and stops hundreds of jetty server instances.

Not every app can be wired up like that, some have external database requirements and the like but even there you often have the ability to write unit tests against an in memory database like derby, which is an excellent approach to compartmentalizing your tests. If you absolutely require an environment for running your tests then many folks have luck with a combination of maven, jetty-maven-plugin and things like selenium but I generally view these as more functional testing, or acceptance testing scenarios, actual unit testing should be done within the context of a junit test...imo at least.

Answer 2

We use a Jenkins build pipeline using the Build Pipeline Plugin . The first job kicks off the build and creates an artifact that includes all the compiled code, including a WAR file and all the test code compiled.

Then, a downstream job is kicked off which deploys the WAR file to a live Tomcat server.

Subsequently we have a job that runs integration tests (getting the tests from the test artifact from the upstream build) against the live Tomcat server.

There's more to it than that, but that is the general idea.