简体繁体中英

Scala RDD[(String,String)] to RDD[String]

原文 2021-08-13 12:44:37 7 1 scala/ apache-spark/ rdd

We are reading a file in UTF_8 as RDD[(String,String)], Need help in converting this to RDD[String].

val textRdd = sparkSession.sparkContext.binaryFiles(filePath,12)
     .mapValues(content => new String(content.toArray(), StandardCharsets.UTF_8))

1 answers

You just need to extract value from tuple ie (string, string).

val outRDD = textRdd.map(t=>t._2) // To get first value use t._1

Scala RDD[String] to RDD[String,String]

Scala RDD String manipulation

Convert RDD[Array[(String,String)]] type to RDD[(String,String)] in scala

scala dataframe to RDD[array[String]]

Scala, RDD Array[string] concatenation

Scala - Turning RDD[String] into a Map

Filter RDD[(VertexId,(VertexId, String, String))] in Scala

convert scala string to RDD[seq[string]]

Convert RDD[(String, String, String)] to RDD[(String, (String, String))] in Spark Scala

Converting String RDD to Int RDD

暂无

暂无

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

Related Question Scala RDD[String] to RDD[String,String] Scala RDD String manipulation Convert RDD[Array[(String,String)]] type to RDD[(String,String)] in scala scala dataframe to RDD[array[String]] Scala, RDD Array[string] concatenation Scala - Turning RDD[String] into a Map Filter RDD[(VertexId,(VertexId, String, String))] in Scala convert scala string to RDD[seq[string]] Convert RDD[(String, String, String)] to RDD[(String, (String, String))] in Spark Scala Converting String RDD to Int RDD

Related Tags

粤ICP备18138465号 © 2020-2024 STACKOOM.COM