[英]Window functions / scala / spark 1.6
I would like to use a window function in Scala. 我想在Scala中使用窗口函数。
I have a CSV file which is the following one : 我有一个CSV文件,它是以下文件:
id;date;value1
1;63111600000;100
1;63111700000;200
1;63154800000;300
When I try to apply a window function over this data frame, sometimes it works and sometimes it fails: 当我尝试在此数据框上应用窗口函数时,有时它可以工作,有时却失败:
val df = loadCSVFile()
val tw = Window.orderBy(date).partitionBy(id).rangeBetween(-5356800000,0)
df.withColumn(value1___min_2_month, min(df.col("value1")).over(tw))
+---+-----------+--------------------+
| id| date|value1___min_2_month|
+---+-----------+--------------------+
| 1|63111600000| 100|
| 1|63111700000| 100|
| 1|63154800000| 100|
+---+-----------+--------------------+
So it works ! 这样就行了! But when I try with a bigger number (which contains row of the previous exemple) I have the following result 但是,当我尝试使用更大的数字(包含上一个示例的行)时,我得到以下结果
val tw =
Window.orderBy(date).partitionBy(id).rangeBetween(-8035200000,0) \n
df.withColumn(value1___min_3_month, min(df.col("value1")).over(tw))
+---+-----------+--------------------+
| id| date|value1___min_3_month|
+---+-----------+--------------------+
| 1|63111600000| null|
| 1|63111700000| null|
| 1|63154800000| null|
+---+-----------+--------------------+
Suffix your number with L
: 在您的电话号码后缀L
:
scala> -10000000000
<console>:1: error: integer number too large
-10000000000
^
scala> -10000000000L
res0: Long = -10000000000
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.