简体   繁体   English

如何展平RDD <List> ?

[英]How to flatten a RDD<List>?

I am fairly new to Spark and using Java to do functional programming with Casandra as datasource. 我对Spark还是相当陌生,并使用Java将Casandra作为数据源进行函数式编程。 As an output of a map task I getting a list of strings. 作为映射任务的输出,我得到了字符串列表。

JavaRDD<List<String>> crows = sessions.map(new Function<Session, List<String>>() {});

I want to convert variable crows from type JavaRDD<List<String>> to JavaRDD<String> ie want to flatten out. 我想将变量乌鸦从JavaRDD<List<String>> to JavaRDD<String>类型转换JavaRDD<List<String>> to JavaRDD<String>即想变平。 Can someone suggest a way to do this? 有人可以建议一种方法吗?

According to @UniLocal the following works: 根据@UniLocal,以下作品:

JavaRDD<String> crows = sessions.flatMap(new FlatMapFunction<Session, String>() {
    public Iterable<String>  call(T x) {
        List<String> someList = ...
        return someList;
    }
});

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM