如何使用Java自定义比较器订购Spark RDD

Question

I have a class that implements a Comparator in this way:

public class MyObject<T>
{
    public static class MyObjectComp<T> implements Comparator<MyObject<T>>
        {
            private LinkedHashSet<Integer> attrList;

            public MyObjectComp (int[] intList)
            {
                this.attrList = new LinkedHashSet<Integer>();
                for (int idx: intList)
                    attrList.add(idx);
            }

            public MyObjectComp (LinkedHashSet<Integer> attrList)
            {
                this.attrList = attrList;
            }


            public int compare(MyObject<T> pf1, MyObject<T> pf2)
            {
                for (Integer idx: attrList)
                {
                    double pf1Norm = pf1.atribute.get(idx).myList.size();
                    double pf2Norm = pf2.atribute.get(idx).myList.size();

                    if (pf1Norm > pf2Norm)
                        return 1;
                    else if (pf1Norm < pf2Norm)
                        return -1;
                }

                return (pf1.key > pf2.key) ? 1 : ((pf1.key < pf2.key) ? -1 : 0); 
            }       

        }   
}

In another part of the code written in scala I created a RDD with this MyObject. Now I need to order the elements of this RDD using this internal comparator class of MyObject. How could I do that using a function like myRDD. sort()? myRDD. sort()?

Answer 1

Something like (note: have not tried this code)

implicit val moo = new MyObjectComp[ T ]( /* ... */ ).asScala

myRDD.sortBy( t => t )