Read csv file into a dataframe and access it using scala

Question

I have a csv file having data such as below -

a1, 1
a2, 2
a3, 3

I want to get output as 1 when I put filter for a1 like filter(a1)._2 .

Answer 1

So the exact syntax would depend on your particular version of Spark. In Spark v2.4.3 you would do this:

val df: DataFrame = sparkSession.sqlContext.read.option("header", 
"false").csv("/path/to/some.csv")

From there you can apply dataframe operations to filter you data:

df.select($"_c1").filter($"_c0" === "a1").show