I'm trying to use a case when in Pyspark and I've found 2 ways to do it:
1st way:
f.expr('count(distinct case when client_ticket==1 then ticket_id else null end)').alias('tickets_clientpay'),
2nd way (now is producing error):
f.when((f.col('client_ticket')==1),f.countDistinct('ticket_id')).alias('tickets_lclientpay'),
I'm personally more confortable with 2nd way, the problem is that I'm getting errors all the time because of the (). Can someone explain me the logic to put () and where? I'm a bit lost and I'm losing time trying attemps all the time.
Thanks!
You can use this syntax. Put countDistinct
outside when
, not inside.
f.countDistinct(
f.when(
f.col('client_ticket') == 1,
f.col('ticket_id')
)
).alias('tickets_lclientpay')
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.