Cassandra的Java Spark流

Question

I'm trying to do java spark streaming with Cassandra. I have done the same with Scala, but I have no clue how to proceed in Java. The web did not give me any examples with java spark streaming and Cassandra.

Could some one please show me how to have the below Scala code in java:

import org.apache.spark.streaming.dstream.ConstantInputDStream

val ssc = new StreamingContext(conf, Seconds(10))

val cassandraRDD = ssc.cassandraTable("mykeyspace", "users").select("fname", "lname").where("lname = ?", "yu")

val dstream = new ConstantInputDStream(ssc, cassandraRDD)

dstream.foreachRDD{ rdd => 
    // any action will trigger the underlying cassandra query, using collect to have a simple output
    println(rdd.collect.mkString("\n")) 
}
ssc.start()
ssc.awaitTermination()

Any help is appreciated. Thanks

Answer 1

In your foreachRDD transformation you can transform your data as per cassandra table format.

JavaRDD<TestBean> cassandraRDD = testRDD
                .flatMap(new FlatMapFunction<Tuple2<String, List<Map<String, Object>>>, TestBean>() {

                    private static final long serialVersionUID = 1L;

                    @Override
                    public Iterable<TestBean> call(Tuple2<String, List<Map<String, Object>>> tuple) throws Exception {

                        return rawData;
                    }
                });

            javaFunctions(jsonRDD).writerBuilder(CASSANDRA_KEYSPACE,CASSANDRA_TABLE, mapToRow(TestBean.class)).saveToCassandra();