与 Scala 2.12 一起使用的 Twitter API

Question

I'm building a Spark (3.0.0) streaming application with Scala 2.12 (built with SBT). How do I get tweets from Twitter API, given all libraries for doing this is for Scala <= 2.11.

EDIT: sample output that I get when trying to built with the libs:

object twitter is not a member of package org.apache.spark.streaming
[error] import org.apache.spark.streaming.twitter.TwitterUtils

Answer 1

In order to import spark-streaming-twitter into your Scala 2.12, you need to specify the Scala version in your import:

resolvers += "twitter_2" at "https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-twitter"
libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.11" % "1.6.3"

This call forces to fetch the scala 2.11 version from artifactory. The import probably tried was:

libraryDependencies += "org.apache.spark" %% "spark-streaming-twitter" % "1.6.3"

The sbt searches for spark-streaming-twitter_2.12 in case your scala build version is 2.12, which does exist in maven atifactory .

Having said that, you need to understand the binary compatibility of scala releases , before you do such thing.

After this import you can call:

import org.apache.spark.streaming.twitter.TwitterUtils