什么时候使用“ sbt汇编”和“ sbt编译&& sbt软件包”？

Question

I would like to know when should I use sbt assembly and when sbt compile && sbt package ?

I'm coding a program in my local computer using Intellij IDEA and I'm compiling it with sbt compile && sbt package , then I'm going to upload it to a cluster and run it using spark-submit . Is this the best way? When should I use sbt assembly ?

Answer 1

I would like to know when should I use sbt assembly and when sbt compile && sbt package ?

Quoting sbt-assembly 's website (which gives sbt assembly ):

The goal is simple: Create a fat JAR of your project with all of its dependencies.

Any dependency in build.sbt (that is not marked as Provided ) is going to be part of the final jar file that is called an uber-jar.

There are at least two reasons why you'd like to use the plugin:

1. Have all dependencies bundled together in one jar file that is much easier to deploy to production (akin to a Docker image)
2. Run your application with one Spark version, say 2.0.0, with a cluster that has spark-submit from Spark 1.6.0

You use sbt compile && sbt package to compile and package the classes of your application (which by the way does not have to use Spark whatsoever). That jar file does not include the dependencies of your application so any dependencies have to be provided in some other way (and makes deployment a bit tricker).

NB: sbt compile && sbt package is just a longer variant of sbt compile package which given that compile task is a dependency of package is just sbt package .

Answer 2

"sbt assembly" is good to use when you want to have uber jar with all dependency included. You will need assembly plugin to use "sbt assembly".

Generally, This kind of jar is good to build when you don't want to provide dependent jars separately on target machine. You can avoid dependent 3rd party jar version mismatch related issues by this kind of packaging.