从 pysapk 中的时间戳列中减去持续时间

Question

I am trying to subtract a column with minutes stored from a timestamp column

And i know the following works for subtracting a fixed duration

from pyspark.sql.functions import current_timestamp, expr

(spark.range(1).select(
    current_timestamp().alias("now"), 
    (current_timestamp() + expr("INTERVAL 12 MINUTES")).alias("now_plus_twelve")))

Is there a way to call this duration value from a column with stored value?

something like this:

(spark.range(1).select(
    current_timestamp().alias("now"), 
    (current_timestamp() + expr("INTERVAL col('duration_column) MINUTES")).alias("now_plus_twelve")))

Answer 1

I figured out this way it works:

(spark.range(1).select(
    current_timestamp().alias("now"), 
    (current_timestamp() + col('duration_column')*expr("INTERVAL 1 minutes")).alias("now_plus_twelve")))