为 API 调用集成进度条

Question

Background: I have seen lots of examples of integrating a progress bar into a for loop, however nothing for my use case, and as such am looking for some advice.

For my use case, I am calling an API and testing if meta is in the response ( meta = data I need). If meta is not in the API response, then the API returns a key pair value named percent_complete , which indicates the data I am trying to return is still aggregating, and provides a value on the progress of data aggregation.

Current code:

def api_call():

    key, secret, url = ini_reader()
    endpoint_url = endpoint_initializer()
    
    while True:
        response = requests.get(url = endpoint_url, auth = HTTPBasicAuth(key, secret), headers = {"vendor-firm": "111"})
        api_response = json.loads(response.text)

        if "meta" not in api_response:
            id_value = "id"
            res1 = [val[id_value] for key, val in api_response.items() if id_value in val]
            id_value = "".join(res1)
            percent_value = "percent_complete"
            res2 = api_response["data"]["attributes"].get("percent_complete", '')*100
            print(f' Your data request for: {id_value} is {res2}% complete!')
            time.sleep(60)

       elif "meta" in api_response:
            
     return api_response

What I am trying to achieve: {res2} *100 gives the percentage, which I would like to use the measure of progress in a progress bar.

Can anyone suggest an appropriate dependency to use?

Answer 1

You can use the Enlighten library. You can keep your print statements and have multiple progress bars at the same time without making any other changes. Below is an example of how you might implement it.

Based on your example it looks like id_value changes, so I wrote the example like that. If it doesn't change you can just use it in the description. And if you have multiples, you'd probably want to create a progress bar for each. If you want to remove your progress bars after they complete, just add leave=False to manager.Counter() .

The library is very customizable and the documentation has a lot ofexamples .

import enlighten

BAR_FORMAT = u'{id_value} {percentage:3.0f}%|{bar}| ' u'[{elapsed}<{eta}, {rate:.2f} %/s]'

manager = enlighten.get_manager()

def api_call():
    pbar = manager.counter(total=100, bar_format=BAR_FORMAT)
    ...
    while True:
        ...
    if "meta" not in api_response:
        ...
        pbar.count = res2
        pbar.update(incr=0, id_value=id_value)

    else:
        ...
        pbar.count = 100
        pbar.update(incr=0, id_value=id_value)

    pbar.close()
    return api_response

Answer 2

Thanks to Aviso, and for everyone's benefit, here is the completed function -

def api_call():
    endpoint_url = endpoint_initializer()
    key, secret, url = ini_reader()
    
    BAR_FORMAT = u'{id_value} {percentage:3.0f}%|{bar}| ' u'[{elapsed}<{eta}, {rate:.2f} %/s]'
    manager = enlighten.get_manager()
    
    date = dt.datetime.today().strftime("%Y-%m-%d")
    print("------------------------------------\n","API URL constructed for:", date, "\n------------------------------------")
    print("-------------------------------------------------------------\n","Endpoint:", endpoint_url, "\n-------------------------------------------------------------") 
    
    pbar = manager.counter(total=100, bar_format=BAR_FORMAT)

    while True:
        response = requests.get(url = endpoint_url, auth = HTTPBasicAuth(key, secret), headers = {"vendor-firm": "381"})
        api_response = json.loads(response.text) 
        
        if "meta" not in api_response:
            id_value = "id"
            res1 = [val[id_value] for key, val in api_response.items() if id_value in val]
            id_value = "".join(res1)
            percent_value = "percent_complete"
            res2 = api_response["data"]["attributes"].get("percent_complete", '')*100
            pbar.count = res2
            pbar.update(incr=0, id_value=id_value)
            time.sleep(60)
        
        elif "meta" in api_response:
            pbar.count = 100
            pbar.update(incr=0, id_value=id_value)
            pbar.close()
            return api_response