![](/img/trans.png)
[英]python write pyspark dataframe to json without header
[英]How to get line without header from dataframe into list with pyspark
我從CSV文件中獲取此數據,我需要將此數據發送到服務器。 但是我只需要這個列表中的值。
{1: Row(Moid=1, Tripid='1', Tstart='2007-05-27', Tend='2007-05-27 08:36:47.846', Xstart='12785', Ystart='1308', Xend='12785', Yend='1308'), 2: Row(Moid=2, Tripid='10', Tstart='2007-05-27', Tend='2007-05-28 08:52:53.673', Xstart='9716', Ystart='-55', Xend='9716', Yend='-55')}
我想得到這個
{ (1, 1, 2007-05-27, 2007-05-2708:36:47.846 , 12785, 1308, 12785, 1308)
(2, 10, 2007-05-27, 2007-05-2808:52:53.673 , 9716, -55, 9716, -55)
您可以使用rdd和一個將行轉換為元組的map函數。 我只是將前三個值用於示例實現:
df = spark.createDataFrame([(1,"1",'2007-05-27'),(2,"10", "2007-05-27")], ['moid',"tripid","tstart"])
print df.rdd.map(lambda r: tuple(r)).collect()
輸出將是一個元組列表:
[(1, u'1', u'2007-05-27'), (2, u'10', u'2007-05-27')]
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.