[英]Error while creating dataframe when querying from cassandra table using spark scala
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext.implicits._
val df =Seq(36,445).toDF()
The above snippet,for creating the dataframe throws the following exception: 上面的代码段用于创建数据框时引发以下异常:
Caused by: `com.datastax.driver.core.exceptions.UnavailableException:` Not enough replicas available for query at consistency LOCAL_ONE (1 required but only 0 ali ve)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:42) ~[dse-java-driver-core-1.2.2.jar:na]
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:29) ~[dse-java-driver-core-1.2.2.jar:na]
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java: 284) ~[dse-java-driver-core-1.2.2.jar:na]
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java: 264) ~[dse-java-driver-core-1.2.2.jar:na]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToM essageDecoder.java:88) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
... 18 common frames omitted
ERROR 2017-11-30 01:52:04,070 org.apache.spark.scheduler.LiveListenerBus: SparkL istenerBus has already stopped! Dropping event SparkListenerExecutorAdded(151202 4724063,0,org.apache.spark.scheduler.cluster.ExecutorData@f32eba1e)
ERROR 2017-11-30 01:52:04,131 org.apache.spark.scheduler.LiveListenerBus: SparkL istenerBus has already stopped! Dropping event SparkListenerBlockManagerAdded(15 12024724131,BlockManagerId(0, localhost, 45074),384093388)
Add this line before you create the dataframe sc.getConf.set("spark.cassandra.input.consistency.level","ANY")
在创建数据
sc.getConf.set("spark.cassandra.input.consistency.level","ANY")
之前添加此行
Here you can check other properties as well... https://docs.datastax.com/en/datastax_enterprise/4.8/datastax_enterprise/spark/sparkCassProps.html 在这里您还可以检查其他属性... https://docs.datastax.com/zh_CN/datastax_enterprise/4.8/datastax_enterprise/spark/sparkCassProps.html
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.