简体   繁体   English

使用Datastax Spark Cassandra Connector将PairDStram写入cassandra

[英]Write PairDStram to cassandra using Datastax Spark Cassandra Connector

I need to write data of my filtered stream to cassandra using Java and Datastax Spark Cassandra Connector. 我需要使用Java和Datastax Spark Cassandra Connector将我的过滤流的数据写入cassandra。

I followed the datastax java documentation . 我按照datastax java文档

The documentation explains how to write a RDD to cassandra, but not how to write a Dstream. 该文档解释了如何将编写RDD写入cassandra,而不是如何编写Dstream。

I need to be able to save a PairDStream and I don't know how to do it, because all examples are written in scala. 我需要能够保存一个PairDStream,我不知道该怎么做,因为所有的例子都是用scala编写的。

I need to turn the following code written in scala in the java's code: 我需要在java的代码中转换以scala编写的代码:

val wc = stream.flatMap(_.split("\\s+"))
    .map(x => (x, 1))
    .reduceByKey(_ + _)
    .saveToCassandra("streaming_test", "words", SomeColumns("word", "count")) 

Basically I have to save a JavaPairDStream<String, Integer> 基本上我必须保存JavaPairDStream<String, Integer>

Solution, in case anyone looking for the answer 解决方案,以防任何人寻找答案

To write a DStream or JavaDStream to Cassandra, need to import: 要将DStream或JavaDStream写入Cassandra,需要导入:

import static com.datastax.spark.connector.japi.CassandraStreamingJavaUtil.*;

and use javaFunctions(DStream<T> arg0) or javaFunctions(JavaDStream<T> arg0) 并使用javaFunctions(DStream<T> arg0)javaFunctions(JavaDStream<T> arg0)

暂无
暂无

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用DataStax Spark Connector在Cassandra中保存空值 - Save null Values in Cassandra using DataStax Spark Connector 使用DatasTax Spark-Cassandra Java连接器运行Spark和Cassandra时出错 - Error running spark and cassandra using datastax spark-cassandra java connector 使用cassandra连接器在apache spark 2.0.2上运行作业时,无法初始化com.datastax.spark.connector.types.TypeConverter $类 - Could not initialize class com.datastax.spark.connector.types.TypeConverter$ while running job on apache spark 2.0.2 using cassandra connector Spark Cassandra 连接器是否有替代品? - Is there an alternative to the Spark Cassandra connector? 无法使用Spark Cassandra连接器1.5.0连接Cassandra 3.0 - Unable to connect Cassandra 3.0 using Spark cassandra connector 1.5.0 使用Spark Cassandra连接器从Cassandra获取数据的问题 - Issues in fetching data from cassandra using spark cassandra connector Spark 和 Cassandra:要求失败:在 com.datastax.spark.connector.japi.CassandraRow 类中找不到列:[mycolumn...] - Spark and Cassandra: requirement failed: Columns not found in class com.datastax.spark.connector.japi.CassandraRow: [mycolumn...] pyspark中的com.datastax.spark:spark-cassandra-connector_2.11:2.5.1软件包出现问题 - Problem with package com.datastax.spark:spark-cassandra-connector_2.11:2.5.1 in pyspark Spark java.lang.NoClassDefFoundError中的spark-cassandra-connector错误:com / datastax / driver / core / ProtocolOptions $ Compression - Error with spark-cassandra-connector in Spark java.lang.NoClassDefFoundError: com/datastax/driver/core/ProtocolOptions$Compression 使用datastax cassandra映射器的NoSuchMethodException - NoSuchMethodException using the datastax cassandra mapper
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM