使用 pyspark 或 pandas 交换列中的单词

Question

I have a dataframe where I want to swap or reverse the order of words in a column per ID.

Input data:

+----------+-----+-------------+
| date     | ID  | words       |
+----------+-----+-------------+
| 09-01-20 | xyz | pixel pearl |
| 09-01-20 | xyz | place order |
| 09-01-20 | xyz | current pro |
| 09-01-20 | xyz | order place |
| 09-01-20 | abc | hello there |
| 09-01-20 | abc | there hello |
| 09-01-20 | abc | very good   |
| 09-01-20 | abc | order place |

Below is the desired result where the 'order place' has been reversed in 4th row. The 1st and 3rd rows remain unchanged since no reverse was present within the same ID 'xyz'. Then for ID 'abc' the order of words will change from 'there hello' to 'hello there', but will do nothing with 'order place'.

+----------+-----+-------------+
| date     | ID  | words       |
+----------+-----+-------------+
| 09-01-20 | xyz | pixel pearl |
| 09-01-20 | xyz | place order |
| 09-01-20 | xyz | current pro |
| 09-01-20 | xyz | place order |
| 09-01-20 | abc | hello there |
| 09-01-20 | abc | hello there |
| 09-01-20 | abc | very good   |
| 09-01-20 | abc | order place |

Thanks! and much appreciated!

Answer 1

In pandas try this:

df['words'] = df.groupby(df['words'].apply(lambda x: ', '.join(set(x.split(' ')))))['words']\
  .transform('first')

Output:

       date   ID        words
0  09-01-20  xyz  pixel pearl
1  09-01-20  xyz  place order
2  09-01-20  xyz  current pro
3  09-01-20  xyz  place order
4  09-01-20  abc  hello there
5  09-01-20  abc  hello there
6  09-01-20  abc    very good
7  09-01-20  abc  place order

Details, first create groups using split and set to get like words together no matter the order. Then, in each group, transform the value in 'words' to get the first occurrence of 'words'.