Scala: Could not find main class when running fat jar

Question

I recently got into some issues after I added two dependencies in my little Scala project. I can no longer run my project. Error: Could not find or load main class abc.DataExtractor . I already spent the entire day looking for similar problems as the same error appeared in quite a lot of posts either here on other platforms.

My fat jar is being created with sbt assembly.

Seq(
  "org.apache.httpcomponents" % "httpclient" % "4.5.12",
  "org.apache.httpcomponents" % "httpmime" % "4.5.12",
  "org.apache.olingo" % "olingo-odata2-api" % "2.0.11",
  "org.apache.olingo" % "olingo-odata2-core" % "2.0.11",
  "com.azure" % "azure-security-keyvault-secrets" % "4.1.0",
  "com.azure" % "azure-identity" % "1.0.5",
  "org.slf4j" % "slf4j-api" % "1.7.30"
  )

As soon as I add the com.azure dependencies, they come witha a lot more of them and I can no longer start up my project.

In my target/scala-2.11 folder I have the two needed jars:

• data-extractor-assembly-0.1.0-SNAPSHOT.jar
• scala-library-2.11.12.jar

In order to start the app, I run the following java -cp data-extractor-assembly-0.1.0-SNAPSHOT.jar:scala-library-2.11.12.jar -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.showdatetime=true -Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG -verbose abc.DataExtractor

My guess is it's got something to do with the build of the assembly. I checked the produced artifact and the MANIFEST.MF file was displaying Main-Class: abc.DataExtractor as expected, but still something is failing. I just can't put my finger of what is the source of the problem.

So, Scala 2.11.12, sbt 1.2.8 and Java 1.8.0_162, if that is relevant

Any information or hints would be greatly appreciated. Thanks.

Answer 1

I did manage to solve this in the end with some sbt assembly magic in the build.sbt file I am also including the scala library files so I only need to specify the fat-jar for the -cp parameter.

assemblyOption in assembly := (assemblyOption in assembly).value.copy(includeScala = true),
assemblyMergeStrategy in assembly := {
  case PathList("javax", "activation", xs @ _*)     => MergeStrategy.last
  case "about.html"                                 => MergeStrategy.rename
  case "plugin.properties"                          => MergeStrategy.last
  case "log4j.properties"                           => MergeStrategy.last
  case "module-info.class" => MergeStrategy.discard
  case PathList("META-INF", xs @ _*) =>
    xs match {
      case ("MANIFEST.MF" :: Nil) => MergeStrategy.discard
      // Concatenate everything in the services directory to keep GeoTools happy.
      case "mailcap" :: Nil => MergeStrategy.last
      case "mimetypes.default" :: Nil => MergeStrategy.last

      case ("services" :: _ :: Nil) =>
        MergeStrategy.concat
      // Concatenate these to keep JAI happy.
      case "javax.media.jai.registryFile.jai" :: Nil | "registryFile.jai" :: Nil |
           "registryFile.jaiext" :: Nil =>
        MergeStrategy.concat
      case name :: Nil => {
        // Must exclude META-INF/*.([RD]SA|SF) to avoid "Invalid signature file digest for Manifest main attributes" exception.
        if (name.endsWith(".RSA") || name.endsWith(".DSA") || name.endsWith(".SF"))
          MergeStrategy.discard
        else
          MergeStrategy.first
      }
      case _ => MergeStrategy.first
    }
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}