How to create new string column extracting integers from a timestamp in Spark?

Question

I have a spark dataframe with a timestamp column, I want a new column which has strings in the format "YYYYMM".

I tried with:

df.withColumn('year_month',year(col("timestamp")).cast("string")+month(col("timestamp")).cast("string"))

But if my timestamp is 2016-10-12, it returns as 2020 as YYYYMM.

Answer 1

You can use date_format :

from pyspark.sql.functions import date_format
df.withColumn('year_month', date_format('timestamp', 'yyyyMM'))