Python 3可迭代的懶惰塊獲取

Question

是否有標准方法能夠延遲獲取下一個數據塊並通過元素產生它

目前，我正在獲取所有塊並將其與itertools鏈接

def list_blobs(container_name:str, prefix:str):    
    chunks = []
    next_marker=None
    while True:
        blobs = blob_service.list_blobs(container_name, prefix=prefix, num_results=100, marker=next_marker)
        next_marker = blobs.next_marker
        chunks.append(blobs)
        if not next_marker:
            break

    return itertools.chain.from_iterable(chunks)

list_blobs提取程序的“懶惰”版本是什么？

Answer 1

您可以只使用yield from ：

def list_blobs(container_name:str, prefix:str):
    next_marker = True
    while next_marker:
        blobs = blob_service.list_blobs(container_name, prefix=prefix, num_results=100, marker=next_marker)
        next_marker = blobs.next_marker
        yield from blobs

Answer 2

用chunks.append(blobs) yield from blobs替換chunks.append(blobs) ，完全擺脫return和chunks list ：

def generate_blobs(container_name:str, prefix:str):
    next_marker = None
    while True:
        blobs = blob_service.list_blobs(container_name, prefix=prefix, num_results=100, marker=next_marker)
        next_marker = blobs.next_marker
        yield from blobs
        if not next_marker:
            break

它將函數轉換為生成器函數，一次生成一個項。

Answer 3

@ ShadowRanger，@Kasrâmvd非常感謝

@timgeb，可通過Azure BlobStorage進行延遲懶散的完整代碼

from azure.storage.blob import BlockBlobService
from azure.storage.blob import Blob
from typing import Iterable, Tuple

def blob_iterator(account:str, account_key:str, bucket:str, prefix:str)-> Iterable[Tuple[str, str]]:
    blob_service = BlockBlobService(account_name=account, account_key=account_key) 

    def list_blobs(bucket:str, prefix:str)->Blob:
        next_marker = None
        while True:
            blobs = blob_service.list_blobs(bucket, prefix=prefix, num_results=100, marker=next_marker)
            yield from blobs
            next_marker = blobs.next_marker
            if not next_marker:
                break

    def get_text(bucket:str, name:str)->str:
        return blob_service.get_blob_to_text(bucket, name).content

    return ( (blob.name, get_text(bucket, blob.name)) for blob in list_blobs(bucket, prefix) )


it = blob_iterator('account', 'account_key', 'container_name', prefix='AA')