从Scala的rdd.map内部进行迭代查找

Question

def retrieveindex (stringlist: List[String], lookuplist: List[String]) = 
  stringlist.foreach(y => lookuplist.indexOf(y))

is my function.

I am trying to use this within an rdd like this:

val libsvm = libsvmlabel.map(x => 
  Array(x._2._2,retrieveindex(x._2._1.toList,featureSet.toList)))

However, I am getting an output that is empty. There is no error, but the output from retrieveindex is empty. When I use println to see if I am retrieving correctly, I do see the indices printed. Is there any way to do this? Should I first 'distribute' the function to all the workers? I am a newbie.

Answer 1

retrieveindex has a return type of type Unit (because of foreach which just applies a function (String) ⇒ Unit on each element). Therefore it does not map to anything.

You probably want it to return the list of indices, like:

def retrieveindex(stringlist: List[String], lookuplist: List[String]): List[Int] = 
  stringlist.map(y => lookuplist.indexOf(y))