Using reference lifetimes correctly in iterator

Question

I am trying to make an iterator out of some struct, s1 , that has references within it with some lifetimes, say 'a and 'b . The iterator elements are new instances of the same struct type holding the same references. When I try to do this, the iterator elements appear to become subject to the lifetime of s1 instead of just 'a and 'b .

An example for clarity:

#[derive(Debug)]
pub struct MyStruct<'a, 'b> {
    num: i32,
    ref1: &'a i32,
    ref2: &'b i32,
}

impl<'a, 'b> MyStruct<'a, 'b> {
    pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct> {
        (1..=n).map(|i| MyStruct { num: self.num + i, ref1: self.ref1, ref2: self.ref2 })
    }
}

fn main() {
    let i1 = 1;
    let i2 = 2;
    let s1 = MyStruct{ num: 0, ref1: &i1, ref2: &i2 };
    let s_next: Vec<_> = s1.with_next_nums(3).collect();
    drop(s1);  // commenting this line the program compiles
    println!("{:?}", s_next);
}

The error I get is:

error[E0505]: cannot move out of `s1` because it is borrowed
  --> src/main.rs:19:10
   |
18 |     let s_next: Vec<_> = s1.with_next_nums(3).collect();
   |                          -------------------- borrow of `s1` occurs here
19 |     drop(s1);  // commenting this line the program compiles
   |          ^^ move out of `s1` occurs here
20 |     println!("{:?}", s_next);
   |                      ------ borrow later used here

For more information about this error, try `rustc --explain E0505`.
error: could not compile `playground` due to previous error

So, because I am dropping s1 , Rust assumes the elements of s_next will become invalid, even though s_next only holds references to i1 and i2 .

I suppose it is a matter of lifetime annotation, but I don't know how to fix it. If I were just producing a single struct out of s1 , then I could annotate the return type like MyStruct<'a, 'b> and it would work, but just using impl Iterator<Item=MyStruct<'a, 'b>> as a return type does not solve it.

Answer 1

The problem is that in this case you need to specify the lifetime of both the iterator elements and the iterator itself. If you just specify the lifetimes of the iterator elements (with impl Iterator<Item=MyStruct<'a, 'b>> ), the compiler assumes you will deal with lifetimes by hand and will give up trying to automatically assign a lifetime to the iterator. However, it does need to have the lifetime of the struct, because it depends on it, so it will fail to compile. The compiler error will actually give you the fix in this case:

error[E0700]: hidden type for `impl Iterator<Item = MyStruct<'a, 'b>>` captures lifetime that does not appear in bounds
  --> src/main.rs:10:9
   |
9  |     pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct<'a, 'b>> {
   |                           ----- hidden type `Map<RangeInclusive<i32>, [closure@src/main.rs:10:21: 10:24]>` captures the anonymous lifetime defined here
10 |         (1..=n).map(|i| MyStruct { num: self.num + i, ref1: self.ref1, ref2: self.ref2 })
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
help: to declare that `impl Iterator<Item = MyStruct<'a, 'b>>` captures `'_`, you can add an explicit `'_` lifetime bound
   |
9  |     pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct<'a, 'b>> + '_ {
   |                                                                                  ++++

For more information about this error, try `rustc --explain E0700`.
error: could not compile `playground` due to previous error

So, you just need to also add that + '_ to the return type to solve the issue:

impl<'a, 'b> MyStruct<'a, 'b> {
    pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct<'a, 'b>> + '_ {
        (1..=n).map(|i| MyStruct { num: self.num + i, ref1: self.ref1, ref2: self.ref2 })
    }
}

Answer 2

The problem with your version is that because of the lifetime elision rules, since you didn't specify the lifetimes for 'a and 'b in the returned MyStruct , the compiler assumes they have the same lifetime as self . So, this as if you wrote:

pub fn with_next_nums<'c>(&'c self, n: i32) -> impl Iterator<Item = MyStruct<'c, 'c>>

To fix that you need to explicitly declare the lifetimes as 'a and 'b :

pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item = MyStruct<'a, 'b>>

Now there will be another error in the declaration of with_next_nums() , because it does borrow from self as the closure captures it. You can fix that by adding '_ , as the compiler suggests, but then your iterator will borrow from self , so your code (that collects them into a Vec ) will work, but similar code that does not collect() immediately will not:

fn main() {
    let i1 = 1;
    let i2 = 2;
    let s1 = MyStruct {
        num: 0,
        ref1: &i1,
        ref2: &i2,
    };
    // The iterator is still alive when we drop `s1`, but it borrows from it.
    let s_next = s1.with_next_nums(3);
    drop(s1);
    let s_next: Vec<_> = s_next.collect();
    println!("{:?}", s_next);
}

A better fix is to change the closure so it will only borrow the fields it needs and not self . You can do it like that:

pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item = MyStruct<'a, 'b>> {
    let MyStruct { num, ref1, ref2 } = *self;
    (1..=n).map(move |i| MyStruct {
        num: num + i,
        ref1,
        ref2,
    })
}