从 PySpark 中的选定列和行中提取值

Question

Spark 3.0

I would like to extract specific values from selected columns into a print function from a spark DF onto my juypter sub-window. I will be doing a for loop so I can automate the monthly files.

So for an example, print('Average salary for a male in company A as an IT is 26000').

I tried x['company'][0][0] for example, but I am not getting the value I needed.

Answer 1

This may be what you're looking for.

df.select('company').collect()[0][0]