如何在 Pyspark 中将字符串更改为时间戳？

Question

I am trying to change a string to a time_stamp in pyspark(Spark version =2.3.0) with below data set and api

I have been trying with different resolution from stack overflow, but nothing could help to change to the time_stamp

df:
|Customer|Transaction_Timestamp|Transaction_Base_Point_Value|
+--------+---------------------+----------------------------+
|Cust1   |10/25/2017 1:47      |2000                        |

Attempt 1

df2 = df.select('Customer', 'Transaction_Timestamp','Transaction_Base_Point_Value', unix_timestamp('Transaction_Timestamp', "dd/MM/yy HH:mm") .cast(TimestampType()).alias("Timestamp")).show(1, False)

Attempt 2

df.withColumn('Time', to_timestamp("Transaction_Timestamp", "yyyy_MM_dd hh_mm_ss").cast("Timestamp"))

Attempt 3

change_type= df.withColumn('Timestamp', col='Transaction_Timestamp').cast('timestamp')

However, the schema produces the following output

 |-- Timestamp: timestamp (nullable = true)

I need to get the output as follows, so that i can perform other operation on timestamp

|Customer|Transaction_Timestamp|Transaction_Base_Point_Value|Timestamp|
+--------+---------------------+----------------------------+---------+
|   Cust1|      10/25/2017 1:47|                        2000|     10/25/2017 1:47|

Answer 1

use to_timestamp from pyspark.sql.functions

.withColumn('Timestamp', to_timestamp('Transaction_Timestamp', 'MM/dd/yyyy hh:mm'))

also a padded hour value would be nice to have not 1:47 but 01:47