如何存储JavaDStream <String> 使用Java在Spark中配置Hive表？

Question

I am receiving the messages from Kafka server into JavaDStream object. Now i want to store the messages into hive table. How can i do that?

Code snippet for receiving messages from Kafka stream:

JavaPairReceiverInputDStream<String, String> kafkaStream = KafkaUtils.createStream(jssc, zookeperIp, consumerGroup,
            topic);

JavaDStream<String> messageData = kafkaStream.map(new Function<Tuple2<String, String>, String>() {

    @Override
    public String call(Tuple2<String, String> message) throws Exception {
        // TODO Auto-generated method stub
        return message._2;
    }

});

Schema for table in Hive:

create external table sample(value STRING);

I am using spark 2.1.0 version.

Answer 1

please take a look at these official programming guides.

You can easily use DataFrames and SQL operations on streaming data

and Spark SQL also supports reading and writing data stored in Apache Hive