Convert DateTime to Date in spark scala

Question

I have following code, trying to change the date format "2010-12-01 8:34" to "2010-12-01" (date format)

val rowRDD = data.map(attributes => Row(attributes(0), attributes(1), attributes(2), attributes(3).toInt, attributes(4), attributes(5).toDouble, attributes(6), attributes(7)))

val r1DF = spark.createDataFrame(rowRDD, schema)

val ts = unix_timestamp($"invoiceDate","yyyy/MM/dd HH:mm").cast("timestamp")

val r2DF = r1DF.withColumn("ts", ts)

r2DF.show()

but the new column ts shows value null in the output. I do not know, what is the problem? Can anybody help?

Answer 1

Try this:

to_date(unix_timestamp($"invoiceDate","yyyy-MM-dd HH:mm").cast("timestamp"))

OR

from_unixtime(unix_timestamp($"invoiceDate","yyyy-MM-dd HH:mm"),"yyyy-MM-dd")