how to solve cyclic-dependency & Iterator lifetime problem?

Question

Rustaceans. when I start to write a BloomFilter example in rust. I found I have serveral problems have to solve. I struggle to solve them but no progress in a day. I need help, any suggestion will help me a lot, Thanks.

Problems

1. How to solve lifetime when pass a Iterator into another function?

// let bits = self.hash(value); // how to solve such lifetime error without use 'static storage?
 
// Below is a workaround code but need to computed in advanced.
let bits = Box::new(self.hash(value).collect::<Vec<u64>>().into_iter());
self.0.set(bits); 

2. How to solve cyclic-dependency between struts without modify lower layer code, eg: bloom_filter ?

// cyclic-dependency:
// RedisCache -> BloomFilter -> Storage
//    |                            ^
//    ------------<impl>------------
//
//                                     v--- cache ownership has moved here
let filter = BloomFilter::by(Box::new(cache));
cache.1.replace(filter);

3. Since rust does not have null value, How can I solve the cyclic-dependency initialization without any stubs?

let mut cache = RedisCache(
    Client::open("redis://localhost").unwrap(),
    // I found can use Weak::new() to solve it,but need to downgrade a Rc reference.
    //                    v-- need a BloomFilter stub to create RedisCache
    RefCell::new(BloomFilter::new()),
);

Code

#![allow(unused)]

mod bloom_filter {
    use std::{hash::Hash, marker::PhantomData};

    pub type BitsIter = Box<dyn Iterator<Item = u64>>;

    pub trait Storage {
        fn set(&mut self, bits: BitsIter);

        fn contains_all(&self, bits: BitsIter) -> bool;
    }

    pub struct BloomFilter<T: Hash>(Box<dyn Storage>, PhantomData<T>);

    impl<T: Hash> BloomFilter<T> {
        pub fn new() -> BloomFilter<T> {
            return Self::by(Box::new(ArrayStorage([0; 5000])));

            struct ArrayStorage<const N: usize>([u8; N]);

            impl<const N: usize> Storage for ArrayStorage<N> {
                fn set(&mut self, bits: BitsIter) {
                    let size = self.0.len() as u64;
                    bits.map(|bit| (bit % size) as usize)
                        .for_each(|index| self.0[index] = 1);
                }

                fn contains_all(&self, bits: BitsIter) -> bool {
                    let size = self.0.len() as u64;
                    bits.map(|bit| (bit % size) as usize)
                        .all(|index| self.0[index] == 1)
                }
            }
        }

        pub fn by(storage: Box<dyn Storage>) -> BloomFilter<T> {
            BloomFilter(storage, PhantomData)
        }

        pub fn add(&mut self, value: T) {
            // let bits = self.hash(value); // how to solve such lifetime error?
            let bits = Box::new(self.hash(value).collect::<Vec<u64>>().into_iter());
            self.0.set(bits);
        }

        pub fn contains(&self, value: T) -> bool {
            // lifetime problem same as Self::add(T)
            let bits = Box::new(self.hash(value).collect::<Vec<u64>>().into_iter());
            self.0.contains_all(bits)
        }

        fn hash<'a, H: Hash + 'a>(&self, _value: H) -> Box<dyn Iterator<Item = u64> + 'a> {
            todo!()
        }
    }
}

mod spi {
    use super::bloom_filter::*;
    use redis::{Client, Commands, RedisResult};
    use std::{
        cell::RefCell,
        rc::{Rc, Weak},
    };

    pub struct RedisCache<'a>(Client, RefCell<BloomFilter<&'a str>>);

    impl<'a> RedisCache<'a> {
        pub fn new() -> RedisCache<'a> {
            let mut cache = RedisCache(
                Client::open("redis://localhost").unwrap(),
                //                    v-- need a BloomFilter stub to create RedisCache
                RefCell::new(BloomFilter::new()),
            );
            //                                      v--- cache ownership has moved here
            let filter = BloomFilter::by(Box::new(cache));
            cache.1.replace(filter);
            return cache;
        }

        pub fn get(&mut self, key: &str, load_value: fn() -> Option<String>) -> Option<String> {
            let filter = self.1.borrow();
            if filter.contains(key) {
                if let Ok(value) = self.0.get::<&str, String>(key) {
                    return Some(value);
                }
                if let Some(actual_value) = load_value() {
                    let _: () = self.0.set(key, &actual_value).unwrap();
                    return Some(actual_value);
                }
            }
            return None;
        }
    }

    impl<'a> Storage for RedisCache<'a> {
        fn set(&mut self, bits: BitsIter) {
            todo!()
        }

        fn contains_all(&self, bits: BitsIter) -> bool {
            todo!()
        }
    }
}

Answer 1

pub type BitsIter = Box<dyn Iterator<Item = u64>>;

In this case, the object in the box must be valid for the 'static lifetime. This isn't the case for the iterator returned by hash - its limited to the lifetime of self .

Try replacing with:

pub type BitsIter<'a> = Box<dyn Iterator<Item = u64> + 'a>;

Or using generics instead of boxed trait objects.

---

So your RedisClient needs a BloomFilter , but the BloomFilter also needs the RedisClient ?

Your BloomFilter should not use the RedisCache that itself uses the BloomFilter - that's a recipe for infinitely recursing calls (how do you know what calls to RedisCache::add should update the bloom filter and which calls are from the bloom filter?).

If you really have to, you need some form of shared ownership, like Rc or Arc . Your BloomFilter will also need to use a weak reference, or else the two objects will refer to each other and will never free.