sbt doesn't correctly deal with resources in multi-module projects when running integration tests. Why?

Question

I have the following configuration on an sbt project:

• moduleA Contains a bunch of integration tests.
• moduleB (depends on moduleA ). Contains a reference.conf file;
• moduleC (aggregates moduleA and moduleB -- this is the root).

When I try to run it:test I get errors, as the tests cannot find the values available on reference.conf . Manually copying the reference.conf to moduleA makes it work.

The issue seems clearly to be that by some reason when running the it:tests (at the root), sbt is not smart enough to add the reference.conf to the classpath.

Can anyone theorize why is that the case? How does sbt work with classpaths and classloaders? Will it just dump everything in a single classloader? It certainly doesn't seem to be the case.

Thanks

Answer 1

In order to address your question and comment, let me break down what SBT is doing with your project.

ModuleC is the root project, and it aggregates ModuleA and ModuleB . In the context of SBT , aggregation means that any command that is run on the root project is also run on the aggregated sub-projects. So, for example, if you run integration tests on the root module, then you will also run the integration tests for its aggregated modules. However, it's important to understand that this is not done all-at-once : the command is run on each sub-project individually.

The next question that SBT has to address is the order in which the sub-projects should be processed. In this case, since ModuleB depends on ModuleA , it has to process ModuleA before processing ModuleB . Otherwise, if there was no dependency between them, then the order wouldn't matter, and SBT would most likely stick with the order that they were specified in ModuleC 's aggregation list.

But what does it mean for one sub-project to depend upon another? It's akin to the relationship between an SBT project and one of its libraryDependencies : the dependent library must be available to the sub-project, and its resources and classes are made available on the classpath during the indicated phases (compile, test, run, etc.).

So, when integration tests are run on ModuleC , SBT will first run ModuleA 's integration tests. Since ModuleA has no other dependencies within the project, it will be processed without any of the other sub-projects available on its classpath . (This is why it cannot access the reference.conf file that is part of ModuleB .) This makes sense, if you think about it, because otherwise—if ModuleA and ModuleB are dependent upon each other—you would have an unresolvable chicken-and-egg situation, in which neither project could be built.

(BTW, if ModuleA has sources that have not yet been compiled, then they will be compiled, on a separate compile pass, before running the integration tests.)

Next it will try to process ModuleB , adding ModuleA 's resources and classes to its classpath, since it is dependent upon them.

From your description, it seems that at least some of the configuration settings in ModuleB 's reference.conf file should belong to ModuleA , since it needs access to them during its integration tests. Whether this means that the whole of the file should belong to ModuleA is up to you. However, it's possible for each sub-project to have its own reference.conf file resource (that's a design feature of the Typesafe Config library that I'm assuming you're using). Any configuration settings that belong to ModuleA 's reference.conf file will also be available to ModuleB , since it is dependent upon ModuleA . (If you have multiple reference.conf files, the only issue you have will depend upon how you package and release ModuleC . If you package everything in all of your sub-projects into a single JAR file, then you would need to merge the various reference.conf files together, for example.)

Another possibility is that some or all of the integration tests should actually belong to ModuleC rather than either ModuleA or ModuleB . Again, making this determination will depend upon your requirements. If it makes sense for each sub-project to perform integration tests in all cases, then place them in the sub-projects. If they only make sense for the completed project as a whole, then put them in ModuleC .

You might want to read the documentation for SBT multi-project builds for further details.