Spring-Boot: How can I build a runnable jar with SBT?

Question

How can I build a runnable jar with SBT? Perhaps Spring-Boot isn't suitable for SBT built applications? Any suggestions on this?

I tried using sbt-assembly but it fails when I try to run it. Note that sbt run works

mainClass in assembly := Some("com.xagongroup.xagon.app.XagonETL")

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", _ @ _*) => MergeStrategy.discard
  case _ => MergeStrategy.first
}

Stack Trace

org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.x.app.XETL]; nested exception is java.lang.Ille
galArgumentException: No auto configuration classes found in META-INF/spring.factories. If you are using a custom packaging, make sure that file is correct.
        at org.springframework.context.annotation.ConfigurationClassParser.processDeferredImportSelectors(ConfigurationClassParser.java:556)
        at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:185)
        at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:308)
        at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:228)
        at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:270)
        at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:93)
        at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:687)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:525)
        at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
        at com.x.app.XETL$.main(XETL.scala:21)
        at com.x.app.XETL.main(XETL.scala)
Caused by: java.lang.IllegalArgumentException: No auto configuration classes found in META-INF/spring.factories. If you are using a custom packaging, make sure that file is correct.
        at org.springframework.util.Assert.notEmpty(Assert.java:277)
        at org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.getCandidateConfigurations(AutoConfigurationImportSelector.java:153)
        at org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.selectImports(AutoConfigurationImportSelector.java:95)
        at org.springframework.context.annotation.ConfigurationClassParser.processDeferredImportSelectors(ConfigurationClassParser.java:547)
        ... 15 common frames omitted

Answer 1

I solved the issue by moving to sbt-native-packager

plugins.sbt

addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.2.0")

build.sbt

scriptClasspath := Seq("*")
mainClass in Compile := Some("com.x.app.XETL")
enablePlugins(JavaAppPackaging)

Running:

1. packaging sbt universal:stage
2. starting the app: target\universal\stage\bin\x.bat

Answer 2

I've solved this without any plugins. It seems the only trick here is to not discard META-INF/spring.factories files in your build.sbt

assemblyMergeStrategy in assembly := {
   case PathList("META-INF", "spring.factories") => MergeStrategy.filterDistinctLines
   case PathList("META-INF", _*) => MergeStrategy.discard
   case _ => MergeStrategy.first
}

Answer 3

In order to save someone else's time:

Brief

Spring Boot uses special jar format which has different inner structure, changes to manifest and can contain jars inside, which should be just stored in resulting archives as-is (it is called ZipEntry.STORED in contrast with ZipEntry.DEFLATED ).

Solution

Neither of plugins I knew (sbt-assembly or sbt-native-packager) could do it from the box or with configuration, so I just wrote a custom task for this which can package my relatively simple project (no modules and subprojects).

File layout

...usual build folders...
project/
    SpringBootJar.scala
build.sbt

build.sbt

Note the addition of spring-boot-loader in the dependencies and springBootJar task.

import SpringBootJar.Keys.springBootJar
import SpringBootJar.springBootJarTask

val springBootVersion = "2.7.6"

lazy val myProject = (project in file("."))
  .settings(
     name := "my-project",
     scalaVersion := "3.2.1",
     ...
     
     libraryDependencies ++= Seq(
       "org.springframework.boot" % "spring-boot-starter-parent" % springBootVersion pomOnly(),
       "org.springframework.boot" % "spring-boot-starter-jdbc" % springBootVersion,
       .... other depenencies ....
       "org.springframework.boot" % "spring-boot-loader" % springBootVersion % "compile"
    ),
    springBootJar := springBootJarTask.value
)

SpringBootJar.scala

import sbt.{Keys, _}
import Keys._
import sbt.io.Using.fileOutputStream
import java.io.BufferedOutputStream
import java.nio.file.Files
import java.util.jar.{Attributes, JarEntry, JarFile, JarOutputStream, Manifest}
import java.util.zip.{CRC32, ZipEntry, ZipOutputStream}
import scala.collection.immutable.TreeSet

object SpringBootJar {
  object Keys {
    val springBootJar = taskKey[Unit]("Create spring boot fat jar")
  }


  val springBootJarTask = Def.task {
    val log = streams.value.log

    // check if spring-boot-loader is already a dependency
    libraryDependencies.value.find(_.name == "spring-boot-loader") match {
      case Some(_) => log.info("spring-boot-loader: present")
      case None => sys.error(s"Consider added spring-boot-loader to libraryDependencies")
    }

    // name of resulting artifact
    val destArtifact = target.value / s"${name.value}-${version.value}-sb.jar"

    val classpath = (Runtime / fullClasspath).value
    val externalClasspath = (Runtime / externalDependencyClasspath).value
    val classDir = (Compile / classDirectory).value
    val resourceDir = (Compile / resourceDirectory).value

    // create directory structure
    val rootDir = IO.createUniqueDirectory(target.value)
    val springBootLoaderDir = rootDir / "org" / "springframework" / "boot" / "loader"
    val appClassesDir = rootDir / "BOOT-INF" / "classes"
    val libsDir = rootDir / "BOOT-INF" / "lib"

    // get the location of the spring-boot-loader library on the file system
    log.info("Seeking for spring-boot-loader jar")
    val springBootLoaderFile = classpath.find(_.data.getName.startsWith("spring-boot-loader")) match {
      case Some(af) => af.data
      case None => sys.error("Couldn't find spring-boot-loader, interrupting")
    }

    // create dir structure
    log.info("Creating directory structure")
    IO.createDirectories(Seq(springBootLoaderDir, appClassesDir, libsDir))

    // creating manifest
    log.info("Creating MANIFEST")
    val manifest = new Manifest()
    val mainAttributes = manifest.getMainAttributes
    mainAttributes.put(Attributes.Name.MANIFEST_VERSION, "1.0")
    mainAttributes.put(Attributes.Name.SPECIFICATION_TITLE, name.value)
    mainAttributes.put(Attributes.Name.SPECIFICATION_VERSION, version.value)
    mainAttributes.put(Attributes.Name.SPECIFICATION_VENDOR, organization.value)
    mainAttributes.put(Attributes.Name.IMPLEMENTATION_TITLE, name.value)
    mainAttributes.put(Attributes.Name.IMPLEMENTATION_VERSION, version.value)
    mainAttributes.put(Attributes.Name.IMPLEMENTATION_VENDOR, organization.value)
    mainAttributes.put(new Attributes.Name("Start-Class"), (Compile / mainClass).value.getOrElse(""))
    mainAttributes.put(Attributes.Name.MAIN_CLASS, "org.springframework.boot.loader.JarLauncher")

    // copy all from org/springframework/boot/loader/ to the same path
    locally {
      log.info("Copying org/springframework/boot/loader/ classes")
      val jar = new JarFile(springBootLoaderFile)
      val entries = jar.entries()
      while (entries.hasMoreElements()) {
        val entry = entries.nextElement()
        if (!entry.isDirectory && entry.getName.startsWith("org/springframework/boot/loader/")) {
          val targetPath = entry.getName.stripPrefix("org/springframework/boot/loader/")
          val targetFile = new File(springBootLoaderDir, targetPath)
          targetFile.getParentFile.mkdirs()
          Files.copy(jar.getInputStream(entry), targetFile.toPath)
        }
      }
      jar.close()
    }

    // copying all own app files
    log.info("Copying app classes")
    IO.copyDirectory(classDir, appClassesDir)

    // copying app resources
    log.info("Copying app resources")
    IO.copyDirectory(resourceDir, appClassesDir)

    // copying all external libs
    log.info("Copying libs")
    IO.copy(externalClasspath.filterNot(_.data.base.startsWith("spring-boot-loader")).map {
      f => f.data -> libsDir / f.data.getName
    })

    // creating jar
    log.info("Creating jar")
    jar(
      (rootDir ** "*").get.map { file => (file, file.relativeTo(rootDir).get.getPath) },
      destArtifact,
      manifest,
      Some(System.currentTimeMillis())
    )

    log.info("Cleaning up")
    IO.delete(rootDir)

    log.info(s"Created: ${destArtifact.getAbsolutePath}")
  }


  private def jar(sources: Traversable[(File, String)], outputJar: File, manifest: Manifest, time: Option[Long]): Unit = {
    val localTime = time.map(t => t - java.util.TimeZone.getDefault.getOffset(t))
    if (outputJar.isDirectory)
      sys.error("Specified output file " + outputJar + " is a directory")
    else {
      val outputDir = outputJar.getParentFile match {
        case null => new File(".")
        case parentFile => parentFile
      }
      IO.createDirectory(outputDir)

      val emptyCRC = new CRC32().getValue // The CRC32 for an empty value, needed to store directories in zip files

      withJarOutput(outputJar, manifest, localTime) { output =>
        writeZip(sources.toSeq, output, localTime) { (file, name) =>
          val entry = new JarEntry(name)
          if (file == null || file.isDirectory) {
            entry.setSize(0)
            entry.setMethod(ZipEntry.STORED)
            entry.setCrc(emptyCRC)
          } else if (file.ext == "jar") {
            val jarBytes = Files.readAllBytes(file.toPath)
            entry.setMethod(ZipEntry.STORED)
            entry.setSize(jarBytes.length)
            entry.setCompressedSize(jarBytes.length)
            entry.setCrc({
              val crc = new CRC32()
              crc.update(jarBytes)
              crc.getValue
            })
          }

          entry
        }
      }
    }
  }


  private def withJarOutput(file: File, manifest: Manifest, time: Option[Long])(f: ZipOutputStream => Unit) = {
    fileOutputStream(false)(file) { fileOut =>
      val zipOut = {
        val os = new JarOutputStream(fileOut)
        val e = new ZipEntry(JarFile.MANIFEST_NAME)
        e.setTime(time.getOrElse(System.currentTimeMillis))
        os.putNextEntry(e)
        manifest.write(new BufferedOutputStream(os))
        os.closeEntry()

        os
      }

      try
        f(zipOut)
      finally
        zipOut.close()
    }
  }


  private def writeZip(sources: Seq[(File, String)], output: ZipOutputStream, time: Option[Long])(
    createEntry: (File, String) => ZipEntry
  ): Unit = {
    val files = sources.flatMap {
        case (file, name) => if (file.isFile) (file, normalizeToSlash(name)) :: Nil else Nil
      }.sortBy {
        case (_, name) => name
      }

    val now = System.currentTimeMillis

    def addDirectoryEntry(file: File, name: String): Unit = {
      output.putNextEntry {
        val e = createEntry(file, name)
        e.setTime(time.getOrElse(now))
        e
      }
      output.closeEntry()
    }

    def addFileEntry(file: File, name: String): Unit = {
      output.putNextEntry {
        val e = createEntry(file, name)
        e.setTime(time.getOrElse(IO.getModifiedTimeOrZero(file)))
        e
      }
      IO.transfer(file, output)
      output.closeEntry()
    }

    // Calculate directories and add them to the generated Zip
    allDirectoryPaths(files).foreach(addDirectoryEntry(null, _))

    // Add all files to the generated Zip
    files foreach { case (file, name) => addFileEntry(file, name) }
  }


  private def normalizeToSlash(name: String): String = {
    val sep = java.io.File.separatorChar
    if (sep == '/') name else name.replace(sep, '/')
  }

  private def relativeComponents(path: String): List[String] =
    path.split("/").toList.dropRight(1)

  private def directories(path: List[String]): List[String] =
    path.foldLeft(List(""))((e, l) => (e.head + l + "/") :: e)

  private def directoryPaths(path: String): List[String] =
    directories(relativeComponents(path)).filter(_.length > 1)

  private def allDirectoryPaths(files: Iterable[(File, String)]) =
    TreeSet[String]() ++ (files flatMap { case (_, name) => directoryPaths(name) })
}

Usage

sbt springBootJar or springBootJar in sbt console.

Conclusion

This was tested with scala 3.2.1 in project and sbt 1.7.1. I don't have time to make it into a plugin, and I don't mind if anyone reuse my findings to make more general and ready to use solution.