Error found when importing spark.implicits

Question

I am using spark 1.4.0

When I tried to import spark.implicits using this command: import spark.implicits._ , this error appear:

<console>:19: error: not found: value spark
   import spark.implicits._
          ^

Can anyone help me to resolve this problem ?

Answer 1

It's because SparkSession is avialable from Spark 2.0 and spark value is an object of type SparkSession in Spark REPL.

In Spark 1.4 use

import sqlContext.implicits._

Value sqlContext is automatically created in Spark REPL for Spark 1.x

Answer 2

To make it complete, first you have to create a sqlContext

import org.apache.spark.{SparkConf, SparkContext}
import org.apache.spark.sql.SQLContext

val conf = new SparkConf().setMaster("local").setAppName("my app")
val sc = new SparkContext(conf)
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._