允許未來在其容器中存儲指向固定值的指針

Question

序幕

我一直在研究這段代碼，它試圖提供一個可回收的 API 來為 REST 分頁器實現異步 stream。

我經歷了多次迭代並決定將 state 存儲在一個描述過程所處的可枚舉中，這既是因為我覺得它最適合這個目的，也是因為它是值得學習的東西，特別明確關於整個過程。 我不想使用stream! 或try_stream! 來自async-stream板條箱。

state 從Begin ，並在使用它發出請求后將PaginationDelegate移動到下一個 state 中。 這個 state 是Pending並擁有delegate和從PaginationDelegate::next_page返回的future 。

當next_page方法需要引用&self時出現問題，但self未存儲在存儲在Pending state 中的未來堆棧幀中。

我想保持這種“扁平化”，因為我發現算法更容易理解，但我也想學習如何以最正確的方式創建這種自引用結構。 我知道我可以包裝未來並讓它擁有PaginationDelegate ，實際上這可能是我最終使用的方法。 盡管如此，我還是想知道如何將這兩個值移動到同一個存儲結構中，並讓指針保持活動狀態以供我自己的教育使用。

---

委托特質

這里定義了一個PaginationDelegate 。 此特征旨在通過 function 的任何方法實現和使用，該方法旨在返回 PaginatedStream 或dyn Stream PaginatedStream 它的目的是定義如何發出請求，以及存儲 state 的有限子集（下一頁從 REST 到 API 的偏移量，以及 API 期望的項目總數）。

#[async_trait]
pub trait PaginationDelegate {
    type Item;
    type Error;

    /// Performs an asynchronous request for the next page and returns either
    /// a vector of the result items or an error.
    async fn next_page(&self) -> Result<Vec<Self::Item>, Self::Error>;

    /// Gets the current offset, which will be the index at the end of the
    /// current/previous page. The value returned from this will be changed by
    /// [`PaginatedStream`] immediately following a successful call to
    /// [`next_page()`], increasing by the number of items returned.
    fn offset(&self) -> usize;

    /// Sets the offset for the next page. The offset is required to be the
    /// index of the last item from the previous page.
    fn set_offset(&mut self, value: usize);

    /// Gets the total count of items that are currently expected from the API.
    /// This may change if the API returns a different number of results on
    /// subsequent pages, and may be less than what the API claims in its
    /// response data if the API has a maximum limit.
    fn total_items(&self) -> Option<usize>;
}

---

Stream State

下一段是enum本身，它作為Stream的實現者和迭代器當前 state 的持有者。

請注意，目前Pending變體將delegate和future分開。 我本可以使用future: Pin<Box<dyn Future<Output = Result<(D, Vec<D::Item>), D::Error>>>>將delegate保留在Future中，但我不想這樣做，因為我想解決根本問題，而不是掩飾它。 此外， delegate字段是一個Pin<Box<D>>因為我正在試驗，我覺得這是我最接近正確的解決方案。

pub enum PaginatedStream<D: PaginationDelegate> {
    Begin {
        delegate: D,
    },
    Pending {
        delegate: Pin<Box<D>>,
        #[allow(clippy::type_complexity)]
        future: Pin<Box<dyn Future<Output = Result<Vec<D::Item>, D::Error>>>>,
    },
    Ready {
        delegate: D,
        items: VecDeque<D::Item>,
    },
    Closed,
    Indeterminate,
}

---

Stream 實施

最后一部分是Stream的實現。 這是不完整的，原因有二； 我還沒有完成它，最好保持示例簡短。

impl<D: 'static> Stream for PaginatedStream<D>
where
    D: PaginationDelegate + Unpin,
    D::Item: Unpin,
{
    // If the state is `Pending` and the future resolves to an `Err`, that error is
    // forwarded only once and the state set to `Closed`. If there is at least one
    // result to return, the `Ok` variant is, of course, used instead.
    type Item = Result<D::Item, D::Error>;

    fn poll_next(mut self: Pin<&mut Self>, ctx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
        // Avoid using the full namespace to match all variants.
        use PaginatedStream::*;

        // Take ownership of the current state (`self`) and replace it with the
        // `Indeterminate` state until the new state is in fact determined.
        let this = std::mem::replace(&mut *self, Indeterminate);

        match this {
            // This state only occurs at the entry of the state machine. It only holds the
            // `PaginationDelegate` that will be used to update the offset and make new requests.
            Begin { delegate } => {
                // Pin the delegate to the heap to ensure that it doesn't move and that pointers
                // remain valid even after moving the value into the new state.
                let delegate = Box::pin(delegate);

                // Set the current state to `Pending`, after making the next request using the
                // pinned delegate.
                self.set(Pending {
                    delegate,
                    future: PaginationDelegate::next_page(delegate.as_ref()),
                });

                // Return the distilled verson of the new state to the callee, indicating that a
                // new request has been made and we are waiting or new data.
                Poll::Pending
            }
            // At some point in the past this stream was polled and made a new request. Now it is
            // time to poll the future returned from that request that was made, and if results are
            // available, unpack them to the `Ready` state and move the delegate. If the future
            // still doesn't have results, set the state back to `Pending` and move the fields back
            // into position.
            Pending { delegate, future } => todo!(),
            // The request has resolved with data in the past, and there are items ready for us to
            // provide the callee. In the event that there are no more items in the `VecDeque`, we
            // will make the next request and construct the state for `Pending` again.
            Ready { delegate, items } => todo!(),
            // Either an error has occurred, or the last item has been yielded already. Nobody
            // should be polling anymore, but to be nice, just tell them that there are no more
            // results with `Poll::Ready(None)`.
            Closed => Poll::Ready(None),
            // The `Indeterminate` state should have only been used internally and reset back to a
            // valid state before yielding the `Poll` to the callee. This branch should never be
            // reached, if it is, that is a panic.
            Indeterminate => unreachable!(),
        }
    }
}

---

編譯器消息

目前，在Begin分支中，有兩條編譯器消息，其中借用delegate ( delegate.as_ref() ) 並將其傳遞給PaginationDelegate::next_page方法。

---

第一個是delegate的壽命不夠長，因為固定值被移動到新的 state 變體Pending中，並且不再駐留在分配給它的 position 中。 我不明白為什么編譯器希望它存在於'static中，如果可以解釋一下，我將不勝感激。

error[E0597]: `delegate` does not live long enough
  --> src/lib.rs:90:59
   |
90 |                     future: PaginationDelegate::next_page(delegate.as_ref()),
   |                             ------------------------------^^^^^^^^^^^^^^^^^-
   |                             |                             |
   |                             |                             borrowed value does not live long enough
   |                             cast requires that `delegate` is borrowed for `'static`
...
96 |             }
   |             - `delegate` dropped here while still borrowed

我還想聽聽您為struct的字段創建值的任何方法，這些字段依賴於應該移入struct的數據（自引用，整篇文章的主要問題）。 我知道在這里使用MaybeUninit是錯誤的（也是不可能的），因為任何稍后會被刪除的占位符值都會導致未定義的行為。 可能給我一個分配未初始化 memory 結構的方法，然后在構建后用值覆蓋這些字段，而不讓編譯器嘗試釋放未初始化的 memory。

---

第二個編譯器消息如下，除了將delegate的臨時值移入結構之外，它與第一個類似。 我理解這基本上是上述相同的問題，但只是通過兩種不同的啟發式方法進行了不同的解釋。 我的理解錯了嗎？

error[E0382]: borrow of moved value: `delegate`
  --> src/lib.rs:90:59
   |
84 |                 let delegate = Box::pin(delegate);
   |                     -------- move occurs because `delegate` has type `Pin<Box<D>>`, which does not implement the `Copy` trait
...
89 |                     delegate,
   |                     -------- value moved here
90 |                     future: PaginationDelegate::next_page(delegate.as_ref()),
   |                                                           ^^^^^^^^^^^^^^^^^ value borrowed here after move

---

環境

這是真實的代碼，但我相信它已經是一個 MCVE。

要為此設置環境，板條箱依賴項如下。

[dependencies]
futures-core = "0.3"
async-trait = "0.1"

以及代碼中使用的導入，

use std::collections::VecDeque;
use std::pin::Pin;
use std::task::{Context, Poll};

use async_trait::async_trait;
use futures_core::{Future, Stream};

Answer 1

下面是我不想使用的潛在解決方案，因為它隱藏了潛在的問題（或者更確切地說，完全避免了這個問題的意圖）。

在定義了PaginatedStream可枚舉的地方，將Pending更改為以下內容。

Pending {
    #[allow(clippy::type_complexity)]
    future: Pin<Box<dyn Future<Output = Result<(D, Vec<D::Item>), D::Error>>>>,
},

現在，在 Stream 的實現中，將Begin的匹配Stream更改為以下內容。

// This state only occurs at the entry of the state machine. It only holds the
// `PaginationDelegate` that will be used to update the offset and make new requests.
Begin { delegate } => {
    self.set(Pending {
        // Construct a new future that awaits the result and has a new type for `Output`
        // that contains both the result and the moved delegate.
        // Here the delegate is moved into the future via the `async` block.
        future: Box::pin(async {
            let result = delegate.next_page().await;
            result.map(|items| (delegate, items))
        }),
    });

    // Return the distilled verson of the new state to the callee, indicating that a
    // new request has been made and we are waiting or new data.
    Poll::Pending
}

編譯器知道那個async塊實際上是async move ，如果你願意，你可以更明確。 這有效地將委托移動到被裝箱和固定的未來堆棧幀中，確保無論何時在 memory 中移動值，這兩個值都會一起移動並且指針不會失效。

另一個匹配 arm 的Pending需要更新以反映簽名的變化。 這是邏輯的完整實現。

// At some point in the past this stream was polled and asked the delegate to make a new
// request. Now it is time to poll the future returned from that request that was made,
// and if results are available, unpack them to the `Ready` state and move
// the delegate. If the future still doesn't have results, set the state
// back to `Pending` and move the fields back into position.
Pending { mut future } => match future.as_mut().poll(ctx) {
    // The future from the last request returned successfully with new items,
    // and gave the delegate back.
    Poll::Ready(Ok((mut delegate, items))) => {
        // Tell the delegate the offset for the next page, which is the sum of the old
        // old offset and the number of items that the API sent back.
        delegate.set_offset(delegate.offset() + items.len());
        // Construct a new `VecDeque` so that the items can be popped from the front.
        // This should be more efficient than reversing the `Vec`, and less confusing.
        let mut items = VecDeque::from(items);
        // Get the first item out so that it can be yielded. The event that there are no
        // more items should have been handled by the `Ready` branch, so it should be
        // safe to unwrap.
        let popped = items.pop_front().unwrap();

        // Set the new state to `Ready` with the delegate and the items.
        self.set(Ready { delegate, items });

        Poll::Ready(Some(Ok(popped)))
    }
    // The future from the last request returned with an error.
    Poll::Ready(Err(error)) => {
        // Set the state to `Closed` so that any future polls will return
        // `Poll::Ready(None)`. The callee can even match against this if needed.
        self.set(Closed);

        // Forward the error to whoever polled. This will only happen once because the
        // error is moved, and the state set to `Closed`.
        Poll::Ready(Some(Err(error)))
    }
    // The future from the last request is still pending.
    Poll::Pending => {
        // Because the state is currently `Indeterminate` it must be set back to what it
        // was. This will move the future back into the state.
        self.set(Pending { future });

        // Tell the callee that we are still waiting for a response.
        Poll::Pending
    }
},