返回对内部集合的引用时的迭代器生命周期问题

Question

I have a struct that lazily load data into an inner vector (but for the example this is ommited). Then I am implementing IntoIterator and Iterator for the IntoIterator type:

struct EntryOwned(u32);
struct Entry<'a>(&'a u32);

impl<'a> EntryOwned {
    fn to_entry(&'a self) -> Entry<'a> {
        Entry(&self.0)
    }
}

struct LazyStruct {
    cache: Vec<EntryOwned>,
}

impl<'a> LazyStruct {
    fn new(data: Vec<EntryOwned>) -> LazyStruct {
        Self {
            cache: data,
        }
    }

    fn get_entry(&'a self, index: usize) -> Option<Entry<'a>> {
        match self.cache.get(index) {
            Some(entry_owned) => Some(entry_owned.to_entry()),
            None => None,
        }
    }
}

impl<'a> IntoIterator for &'a mut LazyStruct {
    type Item = Entry<'a>;
    type IntoIter = LazyStructIter<'a>;

    fn into_iter(self) -> Self::IntoIter {
        LazyStructIter {
            inner: self,
            current_index: 0,
        }
    }
}

struct LazyStructIter<'a> {
    inner: &'a mut LazyStruct,
    current_index: usize,
}

impl<'a> LazyStructIter<'a> {
    fn next_item(&'a mut self) -> Option<Entry<'a>> {
        if self.current_index > self.inner.cache.len() {
            return None;
        }
        let ret = self.inner.get_entry(self.current_index);
        self.current_index += 1;
        ret
    }
}

impl<'a> Iterator for LazyStructIter<'a> {
    type Item = Entry<'a>;

    fn next(&mut self) -> Option<Self::Item> {
        self.next_item()
    }
}

Which yields me to a lifetime issue:

   Compiling playground v0.0.1 (/playground)
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
  --> src/main.rs:64:14
   |
64 |         self.next_item()
   |              ^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 63:5...
  --> src/main.rs:63:5
   |
63 | /     fn next(&mut self) -> Option<Self::Item> {
64 | |         self.next_item()
65 | |     }
   | |_____^
note: ...so that reference does not outlive borrowed content
  --> src/main.rs:64:9
   |
64 |         self.next_item()
   |         ^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 60:6...
  --> src/main.rs:60:6
   |
60 | impl<'a> Iterator for LazyStructIter<'a> {
   |      ^^
note: ...so that the types are compatible
  --> src/main.rs:64:14
   |
64 |         self.next_item()
   |              ^^^^^^^^^
   = note: expected  `&mut LazyStructIter<'_>`
              found  `&mut LazyStructIter<'a>`

The lifetime of the references should be bind to the LazyStruct , but I cannot change the Iter trait do not accept any lifetime specifier. I have already check some answers to similar issues: Iterator returning items by reference, lifetime issue How do I write an iterator that returns references to itself?

EDIT : The Entry is a more complex data structure and it is not possible to copy it. I have to bind it to a specific lifetime since it contains references to things.

One of them points that the iterator should hold a reference to the original collection instead of owning it since it is not allowed to return a reference with the self lifetime. But I could't make it work. So, how should I play the references here?

Here is the playground example

Answer 1

You can't.

You require that each item returned by next() has a reference with a lifetime tied to the existence of the LazyStructIter iterator.

This it is not possible with the Iterator interface:

pub trait Iterator {
    type Item;
    fn next(&mut self) -> Option<Self::Item>;

But it would be possible with a trait defined as:

trait FakeIterator<'a> {
    type Item;

    fn next(&'a mut self) -> std::option::Option<Self::Item>;
}

impl<'a> FakeIterator<'a> for LazyStructIter<'a> {
    type Item = Entry<'a>;

    fn next(&'a mut self) -> Option<Self::Item> {
        self.next_item()
    }
}

Tha capability to have a kind of iterator which returns items borrowing from self is the goal of RFC 1598 - GAT .

See also this article for a collection of workarounds about this topic.

Answer 2

Generally you don't need to specify lifetimes in rust unless 1 of two things is true, the reference is bound to an owner that isn't known, or a reference that is static. Rust says it expected '_ lifetime, which means it understands the ownership and borrows on its own.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c54f5f2f24e635dcfaa0fcbf69fa4ce0

struct EntryOwned(u32);
struct Entry(u32);

impl EntryOwned {
    fn to_entry(&self) -> Entry {
        Entry(self.0)
    }
}

struct LazyStruct {
    cache: Vec<EntryOwned>,
}

impl LazyStruct {
    fn new(data: Vec<EntryOwned>) -> LazyStruct {
        Self {
            cache: data,
        }
    }

    fn get_entry(&self, index: usize) -> Option<Entry> {
        match self.cache.get(index) {
            Some(entry_owned) => Some(entry_owned.to_entry()),
            None => None,
        }
    }
}

impl IntoIterator for LazyStruct {
    type Item = Entry;
    type IntoIter = LazyStructIter;

    fn into_iter(self) -> Self::IntoIter {
        LazyStructIter {
            inner: self,
            current_index: 0,
        }
    }
}

struct LazyStructIter {
    inner: LazyStruct,
    current_index: usize,
}

impl LazyStructIter {
    fn next_item(&mut self) -> Option<Entry> {
        if self.current_index > self.inner.cache.len() {
            return None;
        }
        let ret = self.inner.get_entry(self.current_index);
        self.current_index += 1;
        ret
    }
}

impl Iterator for LazyStructIter {
    type Item = Entry;

    fn next(&mut self) -> Option<Self::Item> {
        self.next_item()
    }
}


fn main() {
    // let json_data = r#"{"item_1" : 10, "item_2" : 100, "item_3" : 1000}"#;
    // if let Ok(data) = serde_json::from_str::<A>(json_data) {
    //     println!("{:?}", data);
    // }
    // else if let Ok(data) = serde_json::from_str::<B>(json_data) {
    //     println!("{:?}", data);
    // }
    for _ in 0..0 {
        println!("foo");    
    }
}

After removing the lifetime specifications, it compiles fine and runs.