sc.textFile之后的zipWithIndex会给出正确的行号吗？

Question

Say If I do, something as shown below.

val rdd = sc.textFile("someFile.txt")
val rddWithLines = rdd.zipWithIndex

Would the indices added by zipWithIndex correspond to the line numbers (first line being 0 of course) in the input file? Or is it that the order gets broken in this case?

Answer 1

zipWithIndex is a map-only transformation (it doesn't shuffle) so order will be correct. You can safely use it here.

Answer 2

SparkContext.textFile can create multiple partitions for each file. If these partitions are in the correct order you should get the correct result. See this answer for more information.