[英]Caused by: java.lang.NoClassDefFoundError: org/apache/spark/sql/types/StructType while it is imported
I'm using Spark SQL dependency just as a data structure already importing it but without passing the SPark context to the Scala's main method.我正在使用 Spark SQL 依赖项,就像已经导入它的数据结构一样,但没有将 SPark 上下文传递给 Scala 的 main 方法。 Why I'm getting this exception?为什么我得到这个异常?
Caused by: java.lang.NoClassDefFoundError: org/apache/spark/sql/types/StructType
the code:编码:
package io.sekai.industry.streaming
import org.apache.spark.sql.types
import org.apache.spark.sql.types.{LongType, StringType, StructField, StructType}
class NonSparkSensorsGeneration {
val schema = StructType(
StructField("id", LongType, nullable = false) :: //always skip this if using the operator
StructField("Energy Data", StringType, nullable = false, metadata = new types.MetadataBuilder().putString("mqtt", "ESP_02/Energy Data").putString("label", "EnergyData").build()) ::
Nil)
def readFromCSVFile(shortPath: String): Unit= {
println("Preparing to load the csv file from jar resource " + shortPath)
val input = getClass.getResourceAsStream(shortPath)
println("Loaded file from resource " + shortPath)
println("Input stream from resource " + shortPath + " details: " + input.toString)
val source = scala.io.Source.fromInputStream(input)
val data = source.getLines.map(_.split("\t")).toArray
source.close
println(data.getClass)
}
def main(args: Array[String]): Unit = {
readFromCSVFile("generation")
}
}
You probably forgot to enable the Add dependencies with "provided" scope to classpath
flag in your IDE (the flag I mentioned is from IntelliJ IDEA)您可能忘记在您的 IDE 中启用将Add dependencies with "provided" scope to classpath
标志的依赖项(我提到的标志来自 IntelliJ IDEA)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.