在pyspark數據框中減去

Question

我想知道subtract原理

target_df = df.subtract(df1)

要么將不在df1的行返回到target_df要么將不在df1 df行返回到target_df

Answer 1

讓我們假設以下示例：

df1 has values as (1,2,3,4,5,6)
df2 has values as (3,4,5,6,7,8)

然后target_df = df1.subtract（df2）的值將為“ df1中的值-兩個dfs中的公共值”，即

(1,2,3,4,5,6) - (3,4,5,6) = (1,2)

請按照以下代碼運行：

from pyspark.sql import Row
df1 = spark.sparkContext.parallelize([Row(1), Row(2), Row(3), Row(4), Row(5), Row(6)]).toDF()
df2 = spark.sparkContext.parallelize([Row(3), Row(4), Row(5), Row(6), Row(7), Row(8)]).toDF()
target_df = df1.subtract(df2)
target_df.show()