Spark Scala Cassandra

Question

Please see the below code and let me know where I am doing it wrong?

Using:

DSE Version - 5.1.0

Connected to Test Cluster at 172.31.16.45:9042. [cqlsh 5.0.1 | Cassandra 3.10.0.1652 | DSE 5.1.0 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help.

Thanks

Cassandra Table :
cqlsh:tdata> select * from map;

 sno | name
-----+------
   1 |  One
   2 |  Two 

-------------------------------------------

scala> :showSchema tdata ======================================== Keyspace: tdata ======================================== Table: map ---------------------------------------- - sno : Int (partition key column) - name : String

scala> val rdd = sc.cassandraTable("tdata", "map")

scala> rdd.foreach(println)

I am not getting anything here? Not even an error.

Answer 1

You have hit a very common spark issue. Your println code is being executed on your remote executor JVMs. That means the printout is to the STDOUT of the executor JVM process. If you want to bring the data back to the driver JVM before printing you need a collect call.

rdd
 .collect //Change from RDD to local collection
 .foreach(println)