如何並行發送 `files_upload_session_append_v2`？ （投遞箱 API，蟒蛇）

Question

我想通過 Dropbox API 以並行方式將一個大文件上傳到 Dropbox（因此上傳速度比順序方式更快）。

文件說

By default, upload sessions require you to send content of the file in
sequential order via consecutive :meth:`files_upload_session_start`,
:meth:`files_upload_session_append_v2`,
:meth:`files_upload_session_finish` calls. For better performance, you
can instead optionally use a ``UploadSessionType.concurrent`` upload
session. To start a new concurrent session, set
``UploadSessionStartArg.session_type`` to
``UploadSessionType.concurrent``. After that, you can send file data in
concurrent :meth:`files_upload_session_append_v2` requests. Finally
finish the session with :meth:`files_upload_session_finish`. There are
couple of constraints with concurrent sessions to make them work. You
can not send data with :meth:`files_upload_session_start` or
:meth:`files_upload_session_finish` call, only with
:meth:`files_upload_session_append_v2` call. Also data uploaded in
:meth:`files_upload_session_append_v2` call must be multiple of 4194304
bytes (except for last :meth:`files_upload_session_append_v2` with
``UploadSessionStartArg.close`` to ``True``, that may contain any
remaining data).

但我不確定如何實現它（因為例如沒有files_upload_async_session_append_v2() ）。 我在 Inte.net 中找不到任何示例。

我嘗試了以下代碼，但與順序代碼相比沒有上傳加速

async def upload_file(local_file_path: str, remote_folder_path: str, client: DropboxBase):
    """
        Uploads a file to Dropbox by chunks. This method uses v2 methods of Dropbox API.

        Example:
            upload_file('test.txt', '/Builds/', dropbox_client)

        :param local_file_path:
        :param remote_folder_path: A path to a folder on Dropbox, must end with a slash.
        :param client: Authorized Dropbox client.
        :return:
    """
    with open(local_file_path, 'rb') as file_stream:
        await __upload_file_by_chunks(file_stream, local_file_path, remote_folder_path, client)

async def test(data: bytes, cursor: UploadSessionCursor, client: DropboxBase, close: bool = False):
    client.files_upload_session_append_v2(data, cursor, close=close)


async def __upload_file_by_chunks(file_stream: BinaryIO, local_file_path: str, remote_folder_path: str, client: DropboxBase):
    # As default size for a chunk 4 MB were chosen. I think it's a good compromise between speed and reliability.
    # Also, Dropbox API guide (https://developers.dropbox.com/dbx-performance-guide) says "Consider uploading chunks in multiples of 4 MBs."
    # ATTENTION: The maximum value can be placed here is 150 MB.
    chunk_size_bytes = 4 * 1024 * 1024

    session_id = __start_upload_session(client)
    cursor = __create_upload_session_cursor(file_stream, session_id)
    file_length = path.getsize(local_file_path)

    test_pool = set()

    # TODO: In theory this can be done in parallel, that should speed up the file upload.
    #  Maybe instead of while loop we can precalculate all chunks and then upload them in parallel.
    while file_stream.tell() < file_length:
        if __chunk_size_is_bigger_than_left_data(file_stream.tell(), file_length, chunk_size_bytes):
            chunk_size_bytes = file_length - file_stream.tell()
            test_pool.add(asyncio.create_task(test(file_stream.read(chunk_size_bytes), cursor, client, close=True)))
            continue

        test_pool.add(asyncio.create_task(test(file_stream.read(chunk_size_bytes), cursor, client)))
        cursor = __create_upload_session_cursor(file_stream, session_id)

    await asyncio.wait(test_pool)

    client.files_upload_session_finish(bytes(), cursor, commit=CommitInfo(
        path=__construct_remote_file_path(local_file_path, remote_folder_path)))


def __start_upload_session(client: DropboxBase) -> int:
    session_start_response = client.files_upload_session_start(bytes(), session_type=UploadSessionType.concurrent)
    return session_start_response.session_id

...

asyncio.run(upload_file(file_name, DROPBOX_TEST_FOLDER, client))

Answer 1

使用“並發”模式是優化使用上傳會話將文件上傳到 Dropbox 的好方法，因為它允許您並行上傳文件的多個部分。

這里有一個代碼示例，展示了如何使用“並發”模式，包括使用files_upload_session_append_v2 ：

https://github.com/dropbox/Developer-Samples/tree/master/Blog/performant_upload