spark：220：错误：“地图”缺少参数类型

Question

Spark 2.1, scala: I am converting GDELT data to GraphX format. However the example listed here fails while creating the hash value, using MurmurHash3: I dont know enough about scala types to diagnose this error message.

 val eventsFromTo = gdelt.select("Actor1Name","Actor2Name").where("actor1Name is not null and actor2name is not null")
    eventsFromTo.show(5)
+-------------+----------+
|   Actor1Name|Actor2Name|
+-------------+----------+
|       SENATE|   RUSSIAN|
|       MEXICO|     TEXAS|
|      RUSSIAN|    SENATE|
|      VERMONT|    CANADA|
|UNITED STATES|    POLICE|
+-------------+----------+
only showing top 5 rows

    val eventActors = gdelt.select("Actor1Name","Actor2Name").where("actor1Name is not null and actor2name is not null").flatMap(x => Iterable(x(0).toString,x(1).toString))
    eventActors.show(5)
+-------+
|  value|
+-------+
| SENATE|
|RUSSIAN|
| MEXICO|
|  TEXAS|
|RUSSIAN|
+-------+

Then I try to convert it for graphX:

val eventVertices: RDD[(VertexId, String)] = eventActors.distinct().map(x => (MurmurHash3.stringHash((x),x)))
<console>:265: error: missing parameter type

If I add a type for i then I get this error:

<console>:265: error: type mismatch;
 found   : String
 required: Int
       val eventVertices: RDD[(VertexId, String)] = eventActors.distinct().map((x:String) => (MurmurHash3.stringHash((x),x)))

Answer 1

I was missing ".rdd" to convert these to RDD's before executing the map()

val eventsFromTo = gdelt.select("Actor1Name","Actor2Name").where("actor1Name is not null and actor2name is not null").rdd
val eventActors = gdelt.select("Actor1Name","Actor2Name").where("actor1Name is not null and actor2name is not null").flatMap(x => Iterable(x(0).toString,x(1).toString)).rdd