Is it a getter in scala? (from the RDD class source of Spark)

Question

When we do checkpointing in spark, we go through a statement that:

checkpointData.get.doCheckPoint()

Why not instead use checkpointData.doCheckPoint() ? Is the get in the statement something like getter? I know that scala class will automatically generate getter and setter.

Or is it some other syntax I do not know?

Answer 1

If you are talking about source code of RDD class ( https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/RDD.scala ), then it's because checkpointData has type of Option[RDDCheckpointData[T]]

See declaration in source code:

private[spark] var checkpointData: Option[RDDCheckpointData[T]] = ...

So to do call a method of RDDCheckpointData we need to get it from Option (after being sure it isDefined , as you can see in code)

Read more about scala Option class: http://www.scala-lang.org/api/current/index.html#scala.Option http://danielwestheide.com/blog/2012/12/19/the-neophytes-guide-to-scala-part-5-the-option-type.html