I have a list of JARs and I want to download the JARs via SBT into destination directory specified. Is there a way/command to do this?
What I am trying is to have a list of jars in classpath for an external system like spark. By default spark adds some jars to classpath and I also have some jars that my app depends on in addition to spark classpath jars.
I don't want to build a fat jar. And I need to package the dependent jars along with my jar in a tar ball.
My build.sbt
name := "app-jar"
scalaVersion := "2.10.5"
dependencyOverrides += "org.scala-lang" % "scala-library" % scalaVersion.value
scalacOptions ++= Seq("-unchecked", "-deprecation")
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.4.1"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.1"
libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka" % "1.4.1"
// I want these jars from here
libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "1.4.0-M3"
libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector-java" % "1.4.0-M3"
libraryDependencies += "com.google.protobuf" % "protobuf-java" % "2.6.1"
...
// To here in my tar ball
So far I have achieved this using a shell script.
I want to know if there is a way to do the same with sbt .
Add sbt-pack to your project/plugins.sbt
(or create it):
addSbtPlugin("org.xerial.sbt" % "sbt-pack" % "0.7.9")
Add packAutoSettings
to your build.sbt
and then run:
sbt pack
In target/pack/lib
you will find all jars (with dependencies).
Update
Add new task to sbt:
val libraries = Seq(
"com.datastax.spark" %% "spark-cassandra-connector" % "1.4.0-M3",
"com.datastax.spark" %% "spark-cassandra-connector-java" % "1.4.0-M3",
"com.google.protobuf" % "protobuf-java" % "2.6.1"
)
libraryDependencies ++= libraries
lazy val removeNotNeeded = taskKey[Unit]("Remove not needed jars")
removeNotNeeded := {
val fileSet = libraries.map(l => s"${l.name}-${l.revision}.jar").toSet
println(s"$fileSet")
val ver = scalaVersion.value.split("\\.").take(2).mkString(".")
println(s"$ver")
file("target/pack/lib").listFiles.foreach{
file =>
val without = file.getName.replace(s"_$ver","")
println(s"$without")
if(!fileSet.contains(without)){
println(s"${file.getName} removed")
sbt.IO.delete(file)
}
}
}
After calling sbt pack
call sbt removeNotNeeded
. You will received only needed jar files.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.