如何转换JavaRDD <list<string> &gt; 到 JavaRDD<string> 并写入没有“[”和“]”的文件</string></list<string>

Question

I have a JavaRDD<List<String>> and my file is getting written with [] at the beginning and end of each list of strings when I use

javacontext.parallelize(rdd).coalesce(1, true).saveAsTextFile("dirname");

Can we convert JavaRDD<List<String>> to JavaRDD<String> and write it to a file?

Answer 1

You could use map to apply String.join for each List<String> in JavaRDD :

String separator = ",";
JavaRDD<String> ys = rdd
        .map(new Function<List<String>, String>() {
            @Override
            public String call(List<String> xs) throws Exception {
                return String.join(separator, xs);
            }
        });

Or using lambdas:

JavaRDD<String> ys = rdd
        .map((Function<List<String>, String>) xs -> String.join(separator, xs));