如何在Windows中使用Scala将Cassandra与Spark连接

Question

I am trying to connect Spark and Cassandra using Scala as described here http://www.planetcassandra.org/blog/kindling-an-introduction-to-spark-with-cassandra/ I am facing errors in the steps under the heading:

"To load the connector into the Spark Shell:"

val test_spark_rdd = sc.cassandraTable(“test_spark”, “test”)

test_spark_rdd.first while using above command(Bold)

it shows the error Exception in task 0.0 in stage 0.0 (TID 0) java.lang.NullPointerException

i have uploaded complete stack trace here

https://docs.google.com/document/d/1UjGXKifD6chq7-WrHd3GT3LoNcw8GawxAPeOtiEjKvM/edit?usp=sharing

Some rpc settings from the cassandra.YAML file are:

rpc_address: localhost 
# rpc_interface: eth1 
# rpc_interface_prefer_ipv6: false 
# port for Thrift to listen for clients on 
rpc_port: 9160 

My spark-defaults config file

# Default system properties included when running spark-submit.
# This is useful for setting default environmental settings.

# Example:
# spark.master                     spark://master:7077
# spark.eventLog.enabled           true
# spark.eventLog.dir               hdfs://namenode:8021/directory
#spark.serializer                 org.apache.spark.serializer.KryoSerializer
#spark.driver.memory              5g
#spark.executor.extraJavaOptions  -XX:+PrintGCDetails -#Dkey=value -Dnumbers="one two three"
spark.cassandra.connection.host localhost

Answer 1

15/08/04 21:24:50 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
    java.lang.NullPointerException
            at java.lang.ProcessBuilder.start(Unknown Source)
            at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
            at org.apache.hadoop.util.Shell.run(Shell.java:418)
            at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
            at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:873)
            at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:853)

Looks like the issue is that the underlying forked executor process failed to start up or do something with the local filesystem. Make sure the default spark directories are accessible by the Executor Process.