[英]Error in Spark Streaming code in scala
I am using spark streaming in scala, where I am working with streaming twitter data. 我正在Scala中使用Spark Streaming,在那里我正在处理Twitter数据。 I have the following code:
我有以下代码:
val ssc = new StreamingContext(new SparkConf(), Seconds(5))
val tweets = TwitterUtils.createStream(ssc, None)
val user = tweets.map(x=> x.getText())
val lang = tweets.map(x=> x.getLang())
I am getting the following error: 我收到以下错误:
[error] /home/user/Lab1.1/Twitterstats.scala:103: value getLang is not a member of twitter4j.Status
[error] val lang = tweets.map(x=> x.getLang())
[error] ^
[error] one error found
What is wrong with the above code? 上面的代码有什么问题? Could someone please help.
有人可以帮忙。
spark-streaming-twitter uses Twitter4j . spark-streaming-twitter使用Twitter4j 。
getLang()
is only supported since version 3.0.6 of Twitter4J . getLang()
因为Twitter4J的3.0.6版本仅支持。 If you are using a version 1.5.2 (or lower) of spark-streaming-twitter you won't be able to call getLang()
because it uses version 3.0.3 of twitter4j . 如果您使用的是1.5.2 (或更低版本)的spark-streaming-twitter ,则将无法调用
getLang()
因为它使用的是twitter4j的 3.0.3版本。 Since 1.6.0 version 4.0.4 is supported as wel as the getLang()
function. 从1.6.0版本开始,与
getLang()
函数一样,都支持4.0.4版本。
So you could upgrade spark-streaming-twitter to 1.6.0 or higher. 因此,您可以将spark-streaming-twitter升级到1.6.0或更高版本。 Or you could use another 3rd party library to detect the language of your tweets.
或者,您可以使用另一个3rd party库来检测您的推文的语言。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.