Why do we need intermediate newBuilder in inits method of Scala List

Question

Scala has inits method defined for List s which does the following:

List(1, 2, 3).inits.toList  // List(List(1, 2, 3), List(1, 2), List(1), List())

It is implemented this way:

def inits: Iterator[Repr] = iterateUntilEmpty(_.init)

private def iterateUntilEmpty(f: Traversable[A @uV] => Traversable[A @uV]): Iterator[Repr] = {
  val it = Iterator.iterate(thisCollection)(f) takeWhile (x => !x.isEmpty)
  it ++ Iterator(Nil) map (x => (newBuilder ++= x).result)
}

I whipped up a less generic version of it (just to play with):

val L = List(1,2,3)
val it = Iterator.iterate(L)(_.init) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) map (x => (new ListBuffer ++= x).result) toList
// List(List(1, 2, 3), List(1, 2), List(1), List())

But then it seemed to me we can do without ListBuffer part:

val L = List(1,2,3)
val it = Iterator.iterate(L)(_.init) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) toList
// List(List(1, 2, 3), List(1, 2), List(1), List())

The result seems to be the same.

So why does the library implementation use newBuilder ?
And how does it affect to performance?

Answer 1

Hopefully, Rex Kerr @rex-kerr will kick off a long-running job of some sort and check SO for questions, but until then, you're asking why doesn't List specialize TraversableLike.inits .

I think the two answers are that specializing for every concrete collection is a maintenance problem, and that overriding the method competes with the JVM's strategy to compile it.

I think I read in the sources that the JVM will tolerate two such dispatches to overrides before performance is affected. I'm not an expert, but the intuition is that assuming a method is effectively final is more efficient.

But wait a sec, your .toList is not the same as .map(_.toList) .

Your question reduces to, why not leave off the builder entirely for List, since you get Lists from the iterator.

Well, maybe it doesn't work out for other types:

scala> val is = (1 to 10).to[Vector]
is: Vector[Int] = Vector(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

scala> val is2 = Iterator.iterate(is)(_.init) takeWhile (x => !x.isEmpty)
is2: Iterator[Vector[Int]] = non-empty iterator

scala> is2 ++ Iterator(Nil)
res9: Iterator[scala.collection.immutable.Seq[Int] with scala.collection.AbstractSeq[Int] with Serializable] = non-empty iterator