如何在 Pyspark 和 Palantir Foundry 中使用多个语句将列的值设置为 0

Question

I'm trying to make a statement that basically says if EDW_ABC.edw_xdpt_act_arrv_lb is NULL then use EDW_ABC.edw_putt_act_arrv_lb and if both are null then set the value to 0. How do I do that? I'm trying the below and I know it's not correct.

EDW_ABC = EDW_ABC.withColumn('act_arrv_abc_lbs', F.when(
(EDW_ABC.edw_xdpt_act_arrv_lb.isNull() == True) & (EDW_ABC.edw_putt_act_arrv_lb.isNull() == True). F.lit(0)\
                                         .otherwise(EDW_ABC.edw_xdpt_act_arrv_lb.isNull()), EDW_ABC.edw_putt_act_arrv_lb)

Answer 1

You don't need to specify a condition in the otherwise , so

EDW_ABC = EDW_ABC.withColumn(
    'act_arrv_abc_lbs',
    F.when(
        EDW_ABC.edw_xdpt_act_arrv_lb.isNull() & EDW_ABC.edw_putt_act_arrv_lb.isNull(), F.lit(0)
    ).otherwise(
        EDW_ABC.edw_putt_act_arrv_lb
    )
)

when and otherwise operate as if & else , so if the first condition in the when isn't satisfied, the otherwise automatically assumes the opposite.

Answer 2

您将需要使用pyspark.sql.functions.coalesce()函数，该函数返回列表中第一列不返回 NULL 的值。

EDW_ABC = EDW_ABC.withColumn('act_arrv_abc_lbs', F.coalesce(F.col("edw_xdpt_act_arrv_lb"), F.col("edw_putt_act_arrv_lb"), F.lit(0))