Maven - dependency conflict with Spark and HtmlUnit

Question

I'd like to use the "spark" web framework (a sinatra clone in Java) in combination with HtmlUnit.

The idea is that my WebService will be able to download websites and parse them (and execute JavaScripts, etc) and then gather some data, do some statistics, etc. HtmlUnit is not just for testing, but actually needed in the main project.

Anyway, Spark runs on Jetty and both Spark and HtmlUnit seem to use the same websocket client library ( org.eclipse.jetty.websocket:websocket-client:9.3.2.v20150730 ), but with different versions. There are a few other libraries as well which seem to make problems.

The projects compiles fine, but fails to launch the web server.

Is there a way to resolve those conflicts somehow?

Here are my dependencies:

<dependencies>
  <dependency>
    <groupId>com.sparkjava</groupId>
    <artifactId>spark-core</artifactId>
    <version>2.3</version>
  </dependency>

   <dependency>
    <groupId>com.sparkjava</groupId>
    <artifactId>spark-template-freemarker</artifactId>
    <version>2.0.0</version>
  </dependency>
  <dependency>
    <groupId>net.sourceforge.htmlunit</groupId>
    <artifactId>htmlunit</artifactId>
    <version>2.19</version>
  </dependency>
</dependencies>

I also found the enforce plugin which lists all the conflicts. Here is the output:

Failed while enforcing releasability the error(s) are [
Dependency convergence error for org.slf4j:slf4j-api:1.7.12 paths to dependency are:
+-com.example:helloworld:1.0-SNAPSHOT
  +-com.sparkjava:spark-core:2.3
    +-org.slf4j:slf4j-api:1.7.12
and
+-com.example:helloworld:1.0-SNAPSHOT
  +-com.sparkjava:spark-core:2.3
    +-org.slf4j:slf4j-simple:1.7.12
      +-org.slf4j:slf4j-api:1.7.12
and
+-com.example:helloworld:1.0-SNAPSHOT
  +-com.sparkjava:spark-template-freemarker:2.0.0
    +-org.slf4j:slf4j-api:1.7.2
,
Dependency convergence error for commons-codec:commons-codec:1.9 paths to dependency are:
+-com.example:helloworld:1.0-SNAPSHOT
  +-net.sourceforge.htmlunit:htmlunit:2.19
    +-org.apache.httpcomponents:httpclient:4.5.1
      +-commons-codec:commons-codec:1.9
and
+-com.example:helloworld:1.0-SNAPSHOT
  +-net.sourceforge.htmlunit:htmlunit:2.19
    +-commons-codec:commons-codec:1.10
,
Dependency convergence error for xml-apis:xml-apis:1.3.04 paths to dependency are:
+-com.example:helloworld:1.0-SNAPSHOT
  +-net.sourceforge.htmlunit:htmlunit:2.19
    +-xalan:xalan:2.7.2
      +-xalan:serializer:2.7.2
        +-xml-apis:xml-apis:1.3.04
and
+-com.example:helloworld:1.0-SNAPSHOT
  +-net.sourceforge.htmlunit:htmlunit:2.19
    +-xerces:xercesImpl:2.11.0
      +-xml-apis:xml-apis:1.4.01
,
Dependency convergence error for org.eclipse.jetty.websocket:websocket-client:9.3.2.v20150730 paths to dependency are:
+-com.example:helloworld:1.0-SNAPSHOT
  +-com.sparkjava:spark-core:2.3
    +-org.eclipse.jetty.websocket:websocket-server:9.3.2.v20150730
      +-org.eclipse.jetty.websocket:websocket-client:9.3.2.v20150730
and
+-com.example:helloworld:1.0-SNAPSHOT
  +-net.sourceforge.htmlunit:htmlunit:2.19
    +-org.eclipse.jetty.websocket:websocket-client:9.2.13.v20150730
,
Dependency convergence error for com.sparkjava:spark-core:2.3 paths to dependency are:
+-com.example:helloworld:1.0-SNAPSHOT
  +-com.sparkjava:spark-core:2.3
and
+-com.example:helloworld:1.0-SNAPSHOT
  +-com.sparkjava:spark-template-freemarker:2.0.0
    +-com.sparkjava:spark-core:2.0.0
]

Answer 1

Exclude dependencies that created conflict in all places except one like this:

<dependency>
    <groupId>com.sparkjava</groupId>
    <artifactId>spark-core</artifactId>
    <version>2.3</version>
    <exclusions>
        <exclusion>
            <artifactId>slf4j-simple</artifactId>
            <groupId>org.slf4j</groupId>
        </exclusion>
    </exclusions>
</dependency>

Add other exclusions wherever it's necessary

EDIT: I created test app with dependencies you listed, and I think this might work

<dependencies>
    <dependency>
        <groupId>com.sparkjava</groupId>
        <artifactId>spark-core</artifactId>
        <version>2.3</version>
        <exclusions>
            <exclusion>
                <artifactId>slf4j-simple</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
            <exclusion>
                <artifactId>websocket-server</artifactId>
                <groupId>org.eclipse.jetty.websocket</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.sparkjava</groupId>
        <artifactId>spark-template-freemarker</artifactId>
        <version>2.0.0</version>
        <exclusions>
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
            <exclusion>
                <artifactId>spark-core</artifactId>
                <groupId>com.sparkjava</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>net.sourceforge.htmlunit</groupId>
        <artifactId>htmlunit</artifactId>
        <version>2.19</version>
        <exclusions>
            <exclusion>
                <artifactId>httpclient</artifactId>
                <groupId>org.apache.httpcomponents</groupId>
            </exclusion>
            <exclusion>
                <artifactId>xalan</artifactId>
                <groupId>xalan</groupId>
            </exclusion>
        </exclusions>
    </dependency>
</dependencies>

Second thing you can use is <dependencyManagement> in root of POM (same level as <dependencies> you used so far). In dependencyManagement you specify which version of which library will be used, like this:

<dependencyManagement>
  <dependencies>
    <dependency>
        <groupId>foo</groupId>
        <artifactId>bar</artifactId>
        <version>1.2.3</version>
    </dependency>
   </dependencies>
</dependencyManagement>

This way, no matter which version of library foo:bar is included in any of dependencies, version 1.2.3 will always be used.

Answer 2

I had the exact same issue and I was able to solve it by setting all the common dependencies for both htmlunit and sparkjava to the same version as follows:

Note that I am using the latest version for both.

  <!-- Spark framework to create restful endpoints -->
        <dependency>
            <groupId>com.sparkjava</groupId>
            <artifactId>spark-core</artifactId>
            <version>2.5.4</version>
        </dependency>

    <!-- HTMLUnit for crawling purposes -->
        <dependency>
            <groupId>net.sourceforge.htmlunit</groupId>
            <artifactId>htmlunit</artifactId>
            <version>2.23</version>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.eclipse.jetty</groupId>
                <artifactId>jetty-webapp</artifactId>
                <version>9.3.6.v20151106</version>
            </dependency>
            <dependency>
                <groupId>org.eclipse.jetty</groupId>
                <artifactId>jetty-util</artifactId>
                <version>9.3.6.v20151106</version>
            </dependency>
            <dependency>
                <groupId>org.eclipse.jetty.websocket</groupId>
                <artifactId>websocket-client</artifactId>
                <version>9.3.6.v20151106</version>
            </dependency>
        </dependencies>
    </dependencyManagement>