Scala's Stream and StackOverflowError

Question

Consider this code (taken from " Functional programming principles in Scala " course by Martin Odersky):

def sieve(s: Stream[Int]): Stream[Int] = {
  s.head #:: sieve(s.tail.filter(_ % s.head != 0))
}

val primes = sieve(Stream.from(2))
primes.take(1000).toList

It works just fine. Notice that sieve is in fact NOT tail recursive (or is it?), even though Stream 's tail is lazy.

But this code:

def sieve(n: Int): Stream[Int] = {
  n #:: sieve(n + 1).filter(_ % n != 0)
}

val primes = sieve(2)
primes.take(1000).toList

throws StackOverflowError .

What is the problem with the second example? I guess filter messes things up, but I can't understand why. It returns a Stream , so it souldn't make evaluation eager (am I right?)

Answer 1

You can highlight the problem with a bit of tracking code:

var counter1, counter2 = 0

def sieve1(s: Stream[Int]): Stream[Int] = {
  counter1 += 1
  s.head #:: sieve1(s.tail.filter(_ % s.head != 0))
}

def sieve2(n: Int): Stream[Int] = {
  counter2 += 1
  n #:: sieve2(n + 1).filter(_ % n != 0)
}

sieve1(Stream.from(2)).take(100).toList
sieve2(2).take(100).toList

We can run this and check the counters:

scala> counter1
res2: Int = 100

scala> counter2
res3: Int = 540

So in the first case the depth of the call stack is the number of primes, and in the second it's the largest prime itself (well, minus one).

Answer 2

Neither one of these are tail recursive.

Using the tailrec annotation will tell you whether or not a function is tail recursive.

Adding @tailrec to the two functions above gives:

import scala.annotation.tailrec

@tailrec
def sieve(s: Stream[Int]): Stream[Int] = {
  s.head #:: sieve(s.tail.filter(_ % s.head != 0))
}

@tailrec
def sieve(n: Int): Stream[Int] = {
  n #:: sieve(n + 1).filter(_ % n != 0)
}

Loading this shows that both definitions are not tail recursive:

<console>:10: error: could not optimize @tailrec annotated method sieve: it contains a recursive call not in tail position
         s.head #:: sieve(s.tail.filter(_ % s.head != 0))
                ^
<console>:10: error: could not optimize @tailrec annotated method sieve: it contains a recursive call not in tail position
         n #:: sieve(n + 1).filter(_ % n != 0)