Spark是否处理资源管理？

Question

I'm new to Apache Spark and I started learning Scala along with Spark. In this code snippet, does Spark handle closing the text file when it is done the program?

val rdd = context.textFile(filePath)

I know in Java when you opened a file you would have to close it with a try-catch-finally or try-with-resources.

In this example, I am mentioning a text file but I want to know if Spark handles closing resources when they are done as RDDs can take multiple different types of data sets.

Answer 1

context.textFile() doesn't actually open the file, it just creates an RDD object. You can verify this experimentally by creating a textFile RDD for a file which doesn't exist- no error will be thrown. The file referenced by the RDD will only be opened, read, and closed when you call an action , which causes Spark to run the IO and data transformations which will result in the action you instructed.