[英]Read csv file into a dataframe and access it using scala
I have a csv file having data such as below -我有一个 csv 文件,其中包含如下数据 -
a1, 1
a1, 1
a2, 2a2, 2
a3, 3a3, 3
I want to get output as 1 when I put filter for a1 like filter(a1)._2 .当我像 filter(a1)._2 那样为 a1 放置过滤器时,我想得到输出为 1 。
So the exact syntax would depend on your particular version of Spark.所以确切的语法将取决于您的特定版本的 Spark。 In Spark v2.4.3 you would do this:
在 Spark v2.4.3 中,你会这样做:
val df: DataFrame = sparkSession.sqlContext.read.option("header",
"false").csv("/path/to/some.csv")
From there you can apply dataframe operations to filter you data:从那里你可以应用数据框操作来过滤你的数据:
df.select($"_c1").filter($"_c0" === "a1").show
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.