简体   繁体   English

Spark dataFrame将列数据类型从字符串转换为日期

[英]Spark dataFrame convert columns Datatype from String to Date

I have the following data, with schema 我有以下数据,有架构

scala> df2.printSchema()
root
 |-- RowID: integer (nullable = true)
 |-- Order Date: string (nullable = true)

scala> df2.show(5)
+-----+----------+
|RowID|Order Date|
+-----+----------+
|    1|   4/10/15|
|   49|   4/10/15|
|   50|   4/10/15|
|   80|   4/10/15|
|   85|   4/10/15|
+-----+----------+

I want to convert the "Order Date" String column to Date data type, and trying the following with no luck, can anyone please suggest a better way to do this? 我想将“订购日期”字符串列转换为日期数据类型,然后尝试以下操作,但没有运气,有人可以建议一种更好的方法吗?

scala> df2.select(df2.col("RowID"), df2.col("Order Date"), date_format(df2.col("Order Date"), "M/dd/yy")).show(5)
+-----+----------+-------------------------------+
|RowID|Order Date|date_format(Order Date,M/dd/yy)|
+-----+----------+-------------------------------+
|    1|   4/10/15|                           null|
|   49|   4/10/15|                           null|
|   50|   4/10/15|                           null|
|   80|   4/10/15|                           null|
|   85|   4/10/15|                           null|
+-----+----------+-------------------------------+

Managed to convert to the unix epoch timestamp, I think from here it is straightforward 设法转换为unix epoch时间戳,我认为从这里开始很简单

scala> df.select(df.col("RowID"), df.col("Order Date"), unix_timestamp(df.col("Order Date"), "M/d/yy")).show(5)
+-----+----------+--------------------------------+
|RowID|Order Date|unixtimestamp(Order Date,M/d/yy)|
+-----+----------+--------------------------------+
|    1|   4/10/15|                      1428604200|
|   49|   4/10/15|                      1428604200|
|   50|   4/10/15|                      1428604200|
|   80|   4/10/15|                      1428604200|
|   85|   4/10/15|                      1428604200|
+-----+----------+--------------------------------+

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用 Scala 将某个 DataType 的所有列的 DataType 转换为 Spark DataFrame 中的另一个 DataType - Convert DataType of all columns of certain DataType to another DataType in Spark DataFrame using Scala 在 spark 中将字符串转换为数据帧的四个不同列 - Convert a string into four different columns of a dataframe in spark for循环中Spark列的数据类型验证-Spark DataFrame - Datatype validation of Spark columns in for loop - Spark DataFrame 根据Apache Spark Scala中的列数据类型将数据框中的列选择为另一个数据框 - Select columns from a dataframe into another dataframe based on column datatype in Apache Spark Scala Spark DataFrame CountVectorizedModel 错误数据类型字符串 - Spark DataFrame CountVectorizedModel Error With DataType String Spark Scala拆分字符串并转换为两列的数据框 - Spark scala split string and convert to dataframe with two columns 在 spark scala 中将日期从字符串转换为日期时间 - Convert Date From String To Datetime in spark scala Spark:将字符串转换为日期 - Spark: convert string to Date 如何在Spark中从数据框的许多列转换为列表? - How to convert to list from many columns of dataframe in Spark? 从数据框中的列获取日期差并获取秒数-Spark Scala - get date difference from the columns in dataframe and get seconds -Spark scala
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM