Convert JavaRDD<List<SomeClass>> to JavaRDD<SomeClass>

Question

I have a JavaRDD of a list of class objects. I want to flatten it to a JaveRDD of class objects such that JavaRDD<{1, 2, 3, 4}, {5, 6, 7}> goes to JavaRDD<1, 2, 3, 4, 5, 6, 7>

in this post Convert RDD List to RDD of individual element in spark one solution is

val newrdd = rdd.flatmap(line => line)

however, line=>line is scala (I think) I tried

rdd.flatmap(line-> line)

and it gives and error

"no instance(s) of type variable(s) U exist so that List conforms to Iterator "

Answer 1

The function you pass to flatMap should return a java.util.Iterator<T> , not a List<T> . It should suffice to do this:

JavaRDD<SomeClass> newRdd = rdd.flatMap(List::iterator);