在PySpark的数据框中获得价值

Question

I have the below dataframe and I'm trying to get the value 3097 as a int, eg storing it in a python variable to manipulate it, multiply it by another int etc.

I've managed to get the row, but I don't even now if it's a good way to do it and I still can't have the value as a int.

data.groupBy("card_bank", "failed").count().filter(data["failed"] == "true").collect()

Answer 1

您需要从序列中获取row （用于循环或映射功能），然后根据https://spark.apache.org/docs/1.4.0/api/java/org/apache row.getInt(2) /spark/sql/Row.html 。

Answer 2

Try selecting the value from spark dataframe :-

df =data.groupBy("card_bank", "failed").count().filter(data["failed"] == "true").collect()
value = df.select("count").as[int].collect()

here, value will be a list.

Answer 3

Get the first record from the Row object using index 0 and get the value using the index "count"

from pyspark.sql.functions import col
data.groupby("card_bank", "failed").count().filter(col("failed") == "true").collect()[0]["count"]