有没有办法将具有列表值的两列组合成一列，其中包含 pyspark dataframe 的列表值

Question

假设我有一个 dataframe 如下：

| id |col 1       |col 2

| 1  | ["A","B"]  |["C","D"]

| 2  | ["A","B"]  |["X","Y"]

| 3  | ["B","D"]  |["N","M"]

我需要一个 output 作为：

| id |merged(col1 and col2)

| 1  | ["A","B","C","D"]

| 2  | ["A","B","X","Y"]

| 3  | ["B","D","N","M"]

Answer 1

您可以使用concat function。

df = df.select('id', F.concat('col1', 'col2').alias('merged(col1 and col2)'))
df.show(truncate=False)

Answer 2

是的，您可以使用 apply -

df['merged'] = df.apply(lambda x: x["col1"] + x["col2"], axis=1)