简体   繁体   中英

Save RDD[Row] as file in scala

I made RDD[Row] data(myData) and try to save it as file

myData.saveAsFile("path")

output
(a, 100, testcontenct)
(b, 200, stackoverflow)

It works well but since it will be treat as csv, I need to take out '(' and ')' symbol. My final goal output is

a, 100, testcontenct
b, 200, stackoverflow

How do I make output file without these symbol.

You could use databricks csv library: https://github.com/databricks/spark-csv

I think it only works on dataframes, but you can easily convert your RDD to a dataframe with

import sqlContext.implicits._
val myDf = myData.toDF

then write it to a file with

myDf.write
    .format("com.databricks.spark.csv")
    .save("path")

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM