Question on pyspark data frame for timestamp
Question
I have a dataframe in which one of the column is a time stamp.
How can i create another data frame from this column in the below format
Can i get some help here please?
1 answers
solution1
0 2019-11-15 12:48:45
Let's say your timestamp column name is time:
from pyspark.sql.functions import *
df2 = df1.withColumn('year',year('time')).withColumn('quarter',quarter('time')).withColumn('weekOfTheYear',weekofyear('time')).withColumn('dayOfTheWeek', dayofweek('time')).withColumn('dayofmonth',dayofmonth('time'))
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
Convert timestamp in pyspark data frame into Jalali date in Python
how to reduce timestamp column value in pyspark data-frame by 1 ms
Pyspark Join data frame
Normalize pyspark data frame by group
Join multiple data frame in PySpark
Group by and filter a Pyspark data frame
Convert Data Frame to string in pyspark
How to transform a timestamp in a data frame
PySpark: Generate timestamp string from available data
Error while inserting data in pyspark data frame
Related Tags