遍歷每一列並找到最大長度

Question

我想從 pyspark dataframe 的每一列中獲取最大長度。

以下是示例 dataframe：

from pyspark.sql.types import StructType,StructField, StringType, IntegerType
data2 = [("James","","Smith","36636","M",3000),
    ("Michael","Rose","","40288","M",4000),
    ("Robert","","Williams","42114","M",4000),
    ("Maria","Anne","Jones","39192","F",4000),
    ("Jen","Mary","Brown","","F",-1)
  ]

schema = StructType([ \
    StructField("firstname",StringType(),True), \
    StructField("middlename",StringType(),True), \
    StructField("lastname",StringType(),True), \
    StructField("id", StringType(), True), \
    StructField("gender", StringType(), True), \
    StructField("salary", IntegerType(), True) \
  ])
 
df = spark.createDataFrame(data=data2,schema=schema)
df.show(truncate=False)

我嘗試實現Scala中提供的解決方案，但無法轉換。
解決方案Scala

我是 Python 的新手，你能幫幫我嗎？

Answer 1

這行得通

from pyspark.sql.functions import col, length, max


df=df.select([max(length(col(name))) for name in df.schema.names])

結果