spark sc.textFile 如何工作？

Question

JavaRDD<String> input = sc.textFile("data.txt");

For the above sample code in Spark, I know it returns distributed list of string. But individual string in that list is a line or word tokens of data.txt?

Answer 1

A string in your rdd equals a line in data.txt.

If the data in your data.txt file is some type of csv data, you can use the spark-csv package that will split the data into columns for you, so you don't have to parse the lines yourself.