[英]“Item does not take parameters” when filtering RDD - scala, Apache Spark
I am working with a csv file in Apache Spark. 我正在使用Apache Spark中的csv文件。 I created the RDD and I want to obtain separate RDDs for each month, so I am filtering by the date, which is expressed as 201601, 201602, etc...
我创建了RDD,我想每月获取单独的RDD,所以我按日期进行过滤,日期表示为201601、201602等。
case class Item(date: String, id: String, classification: String, description: String, algoIndex: String, stratumIndex: String, itemIndex: String, allGmIndex: String, gmRaIndex: String, coicopWeight: String, itemWeight: String, cpihCoicopWeight: String)
val quarter1 = sc.textFile ("examples/Q1.csv")
val q1 = quarter1 map {i => {
val x = i.split(",")
Item(x(0), x(1), x(2), x(3), x(4), x(5), x(6), x(7), x(8), x(9), x(10), x(11))
}
}
val Jan = q1.filter {x=> x(0) == "201601"}
val Feb = q1.filter {x=> x(0) == "201602"}
val Mar = q1.filter {x=> x(0) == "201603"}
The last three lines cause the error "Item does not take parameters", in relation to the following bit: 关于以下位,最后三行导致错误“项目不带参数”:
{x=> x(0) == ..}
How can I fix this? 我怎样才能解决这个问题? What did I do wrong in my class?
我在课堂上做错了什么? Thank you a lot in advance!
提前非常感谢您! :)
:)
You use a case class, so you cannot use apply
syntax. 您使用案例类,因此不能使用
apply
语法。 Use the names you defined: 使用您定义的名称:
q1.filter { x => x.date == "201601" }
or 要么
q1.filter { _.date == "201601" }
Your x
s in the function literals are of type Item
. 函数文字中的
x
属于Item
类型。 Item
does not have an apply(i: Int)
method. Item
没有apply(i: Int)
方法。 Instead, Item
has a bunch of named member variables. 相反,
Item
具有一堆命名成员变量。
Try something like this instead: 尝试这样的事情:
val Jan = q1.filter {x => x.date == "201601"}
It can be abbreviated to: 它可以缩写为:
val Jan = q1.filter (_.date == "201601")
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.