Python-Spark-RDD日期转换

Question

It's possible to convert a RDD element in datetime without transforming the RDD into DataFrame?

I'm trying a lot of approaches but I'm not getting lucky...

data.sortBy(lambda l: to_date(l[-2])).collect()
data.sortBy(lambda l: Date.valueOf(l[-2])).collect()
data.sortBy(lambda l: datetime(l[-2])).collect()

There exists a way to do it?

Thanks!

Answer 1

sortBy can only change the order of elements in an RDD , not their contents. In order to actually get the return values of any one of your datetime conversion functions, you'll have to use map instead:

data.map(lambda l: datetime(l[-2])).collect()