How to use SBT to run ScalaTest tests against a fat jar?

Question

I have a simple SBT project, consisting of some Scala code in src/main/scala and some test code in src/test/scala . I use the sbt-assembly plugin to create a fat jar for deployment onto remote systems. The fat jar includes all the dependencies of the Scala project, including the Scala runtime itself. This all works great.

Now I'm trying to figure out a way I can run the Scala tests against the fat jar. I tried the obvious thing, creating a new config extending the Test config and modifying the dependencyClasspath to be the fat JAR instead of the default value, however this fails because (I assume because) the Scala runtime is included in the fat jar and collides somehow with the already-loaded Scala runtime.

My solution right now works but it has serious drawbacks. I just use Fork.java to invoke Java on the org.scalatest.tools.Runner runner with a classpath set to include the test code and the fat jar and all of the test dependencies. The downside is that none of the SBT test richness works, there's no testQuick , there's not testOnly , and the test failure reporting is on stdout.

My question boils down to this: how does one use SBT's test commands to run tests when those tests are dependent not on their corresponding SBT compile output, but on a fat JAR file which itself includes all the Scala runtimes?

Answer 1

This is what I landed on (for specs2, but can be adapted). This is basically what you said was your Fork solution, but I figured I'd leave this here in case someone wanted to know what that might be. Unfortunately I don't think you can run this "officially" as a SBT test runner. I should also add that you still want Fork.java even though this is Scala, because Fork.scala depends on a runner class that I don't seem to have.

test.sbt (or build.sbt, if you want to put a bunch of stuff there - SBT reads all .sbt files in the root if you want to organize):

// Set up configuration for building a test assembly
Test / assembly / assemblyJarName := s"${name.value}-test-${version.value}.jar"
Test / assembly / assemblyMergeStrategy := (assembly / assemblyMergeStrategy).value
Test / assembly / assemblyOption := (assembly / assemblyOption).value
Test / assembly / assemblyShadeRules := (assembly / assemblyShadeRules).value
Test / assembly / mainClass := Some("org.specs2.runner.files")
Test / test := {
  (Test / assembly).value

  val assembledFile: String = (Test / assembly / assemblyOutputPath).value.getAbsolutePath
  val minimalClasspath: Seq[String] = (Test / assembly / fullClasspath).value
    .filter(_.metadata.get(moduleID.key).get.organization.matches("^(org\\.(scala-lang|slf4j)|log4j).*"))
    .map(_.data.getAbsolutePath)

  val runClass: String = (Test / assembly / mainClass).value.get
  val classPath: Seq[String] = Seq(assembledFile) ++ minimalClasspath
  val args: Seq[String] = Seq("-cp", classPath.mkString(":"), runClass)

  val exitCode = Fork.java((Test / assembly / forkOptions).value, args)

  if (exitCode != 0) {
    throw new TestsFailedException()
  }
}
Test / assembly / test := {}

Change in build.sbt:

lazy val root = (project in file("."))
  .settings(/* your original settings are here */)
  .settings(inConfig(Test)(baseAssemblySettings): _*) // enable assembling in test

Answer 2

Try adding the fat jar to the first position on the test classpath like so

Test / fullClasspath := Attributed.blank((assembly / assemblyOutputPath).value) :: (Test / fullClasspath).value.toList

Here assemblyOutputPath gives the path to the fat jar. My understanding is JVM loads the first class it encounters on the classpath, so in effect we would be testing of the fat jar.

We could capture this idea in a custom command testAgainstFatJar like so

commands += Command.command("testAgainstFatJar") { state =>
    "set assembly / test := {}" ::
    "assembly" ::
    "set Test / fullClasspath := Attributed.blank((assembly / assemblyOutputPath).value) :: (Test / fullClasspath).value.toList" ::
    "test" :: state
}

Check the fat jar is indeed in the first position with show Test / fullClasspath