如何通过在python中添加2个RDD的对应元素来创建RDD

Question

So I have 2 RDDs (lets say RDD1 and RDD2), each with a list of numbers. The two lists are of the same size. I want to create a RDD3 where each element in RDD3 is the addition of the corresponding elements of RDD1 and RDD2. How do I do this in python using pyspark functions?

Answer 1

If the list are not too big then following can work. Let me know if this works or if you have other suggestions

rdd1 = sc.parallelize([100,200,300])
rdd2 = sc.parallelize([101,202,303])
print(rdd1.collect())
print(rdd2.collect())
# [100, 200, 300]
# [101, 202, 303]
output = []
for i, element in enumerate(rdd1.collect()):
  output.append(element + rdd2.collect()[i])
rdd3 = sc.parallelize(output)
print(rdd3.collect())
# [201, 402, 603]