Mergesort使用apache-spark

Question

I am new to Apache Spark, I have to write java code for mergesort in apache spark.

SparkConf conf = new SparkConf().setMaster("local").setAppName("Merge Sort App");
JavaSparkContext sc = new JavaSparkContext(conf);

JavaRDD<Integer> input = sc.parallelize(a); // where a is list of Integers  

I am done with loading array-list of Integers in javaRDD<Integer> :

I have to use integer element list to sort using megersort with apache spark.

Any help?

Answer 1

You could define a function merge that takes in two lists of type Integer and outputs a single sorted list of Integer 's. Then just call input.reduce(<classname>::merge) .

Of course, this depends on you having your RDD hold lists of Integer s instead of individual elements. Think the first part of merge sort where you recurse to the single elements. You could do this by writing a method toList that takes an Integer and outputs a List with that element in it. Then you can just do input.map(<classname>::toList).reduce(<classname>::merge) .