Reading rows from Hive table and write to file in Scala-Spark

Question

I want to read data rows from one of the hive table in spark-scala program.After this , same data needs to write into a file row by row . Actually write to file row by row. Could anyone share pointers? Spark version 1.6,Hive1.2.

Answer 1

You can read from the table like so...

val mydf = hiveContext.sql("select * from hive_table_name")

mydf.write.save.format("com.databricks.spark.csv").option("header", "true").save(hdfs_path_to_save)

here is help with the csv parser that you need if it's before Spark 2.0 https://github.com/databricks/spark-csv