[英]could not find implicit value for evidence parameter of type - Spark
Im new to Scala and spark and could do with some help regarding the above error. 我是Scala和Spark的新手,可以针对上述错误提供一些帮助。 Here is a snippet of my code that is causing issues:
这是导致问题的代码片段:
case class Session (user_id: String, creation_date: BigInt, offline: Boolean)
case class User (user_id: String, app_id: Int, vendor_code: String, app_version: String)
val users = sc.cassandraTable[User]("leech_seed", "user").select("user_id", "app_id", "vendor_code", "app_version").where("last_active >=" + (timestamp - 86400000))
val sessions = sc.cassandraTable[Session]("leech_seed", "session").select("user_id", "creation_date", "offline").where("creation_date < " + timestamp + " AND creation_date >=" + (timestamp - 86400000))
when i use this code in the spark shell it works fine but when i am trying to build a jar with sbt i get the following error could not find implicit value for evidence parameter of type com.datastax.spark.connector.rdd.reader.RowReaderFactory[User]
当我在Spark Shell中使用此代码时,它工作正常,但是当我尝试使用sbt构建jar时,出现以下错误
could not find implicit value for evidence parameter of type com.datastax.spark.connector.rdd.reader.RowReaderFactory[User]
This has been doing my head in for longer than id like to admit so any help/insight would be greatly appreciated. 这样做的时间比我想承认的要长得多,因此我们将不胜感激任何帮助/见解。
Note: I am using the datastax cassandra connector for spark 注意:我正在使用datastax cassandra连接器来产生火花
Check your spark-cassandra connector version is up-to-date with the version of Spark you are using. 检查您的spark-cassandra连接器版本是否与您所使用的Spark版本保持最新。 I have encountered these issues using connector versions older than 2.10-1.4.0-M3 with Spark 1.4.1.
我在使用Spark 1.4.1和2.10-1.4.0-M3之前的连接器版本时遇到了这些问题。
Also ensure that your case classes are defined outside of your main method - else you will encounter No RowReaderFactory can be found for this type
or similar. 还要确保您的case类是在main方法之外定义的-否则您将遇到
No RowReaderFactory can be found for this type
或类似No RowReaderFactory can be found for this type
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.