从 function 返回一个泛型类型，在 rust 中没有泛型参数

Question

I'm currently trying to write a small function in Rust that returns a iterator over the tokens of a simple LISP-style calculator language. I ran into a compile error when I didn't expect to.

My first attempt to write the function was:

fn tokenizer_for<'a, I>(s: &'a str) -> Peekable<I> where I: Iterator<Item=&'a str> {
    s.split_whitespace()
        .flat_map(
            |word| {
                word.replace("(", "( ").replace(")", " )").split_whitespace()
            }
        )
        .peekable()
}

However rustc replies:

error[E0308]: mismatched types
  --> src/lib.rs:4:5
   |
3  |   fn tokenizer_for<'a, I>(s: &'a str) -> Peekable<I> where I: Iterator<Item=&'a str> {
   |                        -                 ----------- expected `std::iter::Peekable<I>` because of return type
   |                        |
   |                        this type parameter
4  | /     s.split_whitespace()
5  | |         .flat_map(
6  | |             |word| {
7  | |                 word.replace("(", "( ").replace(")", " )").split_whitespace()
8  | |             }
9  | |         )
10 | |         .peekable()
   | |___________________^ expected type parameter `I`, found struct `std::iter::FlatMap`
   |
   = note: expected struct `std::iter::Peekable<I>`
              found struct `std::iter::Peekable<std::iter::FlatMap<std::str::SplitWhitespace<'_>, std::str::SplitWhitespace<'_>, [closure@src/lib.rs:6:13: 8:14]>>`
   = help: type parameters must be constrained to match other types
   = note: for more information, visit https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters

I found that using impl in the return type worked (which I prefer):

fn tokenizer_for(s: &str) -> Peekable<impl Iterator<Item=&str>> {
    s.split_whitespace()
        .flat_map(
            |word| {
                word.replace("(", "( ").replace(")", " )").split_whitespace()
            }
        )
        .peekable()
}

But I had expected to be able to choose between either option, even though it appears that the latter attempt that worked might not even be producing a generic function.

Why can I not use a where clause to specify a generic type in the former case?

I have used where clauses to constrain generic parameters that appear in return types before. Does that only work when the generic parameter also appears in the function's parameters?

Any references that explain more about the details of this distinction would be particularly helpful.

Answer 1

In Rust, when you have lifetimes or type parameters specified in the Item<...> list, those lifetimes or type parameters are chosen at the use site.

This function signature says that the caller of the function can choose what type I is:

fn tokenizer_for<'a, I>(s: &'a str) -> Peekable<I> where I: Iterator<Item=&'a str>;

but that doesn't compile because the return type is actually the return type of the call to Iter::peekable() (ie, the type from your error message std::iter::Peekable<std::iter::FlatMap<std::str::SplitWhitespace<'_>, std::str::SplitWhitespace<'_>, [closure@src/lib.rs:6:13: 8:14]>> ).

On the other hand, this signature says that the return type is merely something that implements Iterator<Item=&str>> :

fn tokenizer_for(s: &str) -> Peekable<impl Iterator<Item=&str>>;

The caller cannot choose what type that is; the compiler infers the actual type from the function body.