Why iterators start raising StopIteration instead of my custom exception?

Question

I have an iterator which may raise a custom exception.

def raising_iter():
    yield 123
    while True:
        raise ValueError("My custom exception")


it = raising_iter()

# prints 123
print(next(it))

try:
    next(it)
except ValueError as e:
    print(e)

# raises StopIteration
next(it)

After it has raised that exception, the subsequent calls to next() raise StopIteration, despite the iterator is not exhausted and it hasn't raised StopIteration before. It seems the rule is like this: "If the iterator has raised anything , it will raise StopIteration on the subsequent calls".

1st question Why? The docs don't mention this behaviour. They only say

Once an iterator’s next () method raises StopIteration, it must continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken.

2nd question How to make the iterator raise my custom exception? The rationale is to communicate to the clients the exact reason why the iteration stopped. One workaround is to define my own iterator protocol. It only needs to have one method - next() , but it won't have that undesired behaviour. This is problematic, though, as I need to rewrite every other part of my code which expects a normal iterator.

I understand that this breaks LSP and thus it's a misuse of iterator (although, I didn't expect such a harsh punishment from Python). Perhaps, my use case requires some different solution?

Tested with Python 3.7

Answer 1

I came up with a somewhat decent solution. Instead of yielding values and raising exceptions, I will yield a special wrapper which can have either a value or an exception inside.

import abc
from typing import TypeVar, Generic, Optional, Iterator

T = TypeVar("T")


class Wrapped(Generic[T]):
    def __init__(
            self,
            value: Optional[T] = None,
            exception: Optional[BaseException] = None
    ):
        self._value = value
        self._exception = exception
        if value is not None and exception is not None:
            raise ValueError()

    def unwrap(self) -> T:
        if self._exception:
            raise self._exception
        return self._value

    @classmethod
    def with_value(cls, value: T) -> 'Wrapped[T]':
        return cls(value=value)

    @classmethod
    def with_exception(cls, exception: BaseException) -> 'Wrapped':
        return cls(exception=exception)


class WrappedItemsIterator(Iterator[Wrapped[T]], Generic[T]):
    def __iter__(self):
        return self

    def __next__(self) -> Wrapped[T]:
        try:
            value = self._real_next()
        except BaseException as e:
            return Wrapped.with_exception(e)
        else:
            return Wrapped.with_value(value)

    @abc.abstractmethod
    def _real_next(self) -> Wrapped[T]:
        raise NotImplementedError()


class MyIterator(WrappedItemsIterator):
    def _real_next(self) -> Wrapped[T]:
        raise ValueError("Custom exception")


it = MyIterator()
value = next(it).unwrap()

When you call unwrap() that will either return a value or raise an exception.