如何重试 x 次并打印长时间睡眠

Question

I have been trying to understand how backoff works. My goal is that whenever I reach a status_code etc: 405 5 times. I want to put a sleep of 60000 sec and print out that there has occured status error 405.

Right now I have written:

import time

import backoff
import requests

@backoff.on_exception(
    backoff.expo,
    requests.exceptions.RequestException,
    max_tries=5,
    giveup=lambda e: e.response is not None and e.response.status_code == 405
)
def publish(url):
    r = requests.post(url, timeout=10)
    r.raise_for_status()


publish("https://www.google.se/")

and what happens now is that if it just reaches 405 once, it will raise the status_code and stop the script. What im looking for is how I can make the script to retry 5 times, if the status is 5 times in a row of 405, then we want to put a long sleep and print it out. How can I do that using backofF? Im also up for other suggestions:)

The old way of counter:

    import requests
    import time
    from requests.exceptions import ConnectionError, ReadTimeout, RequestException, Timeout
    
    exception_counter = 0
    
    while True:
    
        try:
            response = requests.get("https://stackoverflow.com/", timeout=12)
    
            if response.ok:
                print("Very nice")
                time.sleep(60)
    
            else:
                print(
                    f'[Response -> {response.status_code}]'
                    f'[Response Url -> {response.url}]'
                )
                time.sleep(60)
    
                if response.status_code == 403:
                    if exception_counter >= 10:
                        print("Hit limitation of counter: Response [403]")
                        time.sleep(4294968)
    
                    exception_counter += 1
    
        
    except (ConnectionError) as err:
        print(err)
        time.sleep(random.randint(1, 3))
        
        if exception_counter >= 10:
            print(f"Hit limitation of coonnectionerror {err}")
            time.sleep(4294968)
            continue
        
        exception_counter += 1
        continue
        
    except (ReadTimeout, Timeout) as err:
        print(err)
        time.sleep(random.randint(1, 3))
        continue

    except RequestException as err:
        print(err)
        time.sleep(random.randint(1, 3))
        continue

    except Exception as err:
        print(err)
        time.sleep(random.randint(1, 3))
        
        if exception_counter >= 10:
            print(f"Hit limitation of Exception {err}")
            time.sleep(4294968)
            continue
        
        exception_counter += 1
        continue

Answer 1

You didn't say what you want to do after sleeping for 60000 seconds, so I set it up to sleep after four attempts, and then do one final (fifth) attempt before failing proper.

You can add custom logic like you've asked for with an on_backoff handler.

Also I rejigged your giveup function, you might have had the boolean around the wrong way.

import time
import backoff
import requests


def backoff_hdlr(details):
    print("backoff_hdlr", details)
    if details["tries"] >= 4:
        print(f"sleeping")
        time.sleep(1)  # 60000


@backoff.on_exception(
    backoff.expo,
    requests.exceptions.RequestException,
    max_tries=5,
    giveup=lambda e: e.response.status_code != 405,
    on_backoff=backoff_hdlr,
)
def publish(url):
    print(f"called publish with url={url}")
    r = requests.post(url, timeout=10)
    r.raise_for_status()


publish("https://www.google.se/")

/Users/michael/.conda/envs/mip_typing/bin/python /Users/michael/git/mip_typing/scratch_2.py
called publish with url=https://www.google.se/
backoff_hdlr {'target': <function publish at 0x7fe45c626b80>, 'args': ('https://www.google.se/',), 'kwargs': {}, 'tries': 1, 'elapsed': 1.5e-05, 'wait': 0.8697943681459608}
called publish with url=https://www.google.se/
backoff_hdlr {'target': <function publish at 0x7fe45c626b80>, 'args': ('https://www.google.se/',), 'kwargs': {}, 'tries': 2, 'elapsed': 1.144912, 'wait': 1.5425500028676453}
called publish with url=https://www.google.se/
backoff_hdlr {'target': <function publish at 0x7fe45c626b80>, 'args': ('https://www.google.se/',), 'kwargs': {}, 'tries': 3, 'elapsed': 2.949183, 'wait': 0.2052666718206697}
called publish with url=https://www.google.se/
backoff_hdlr {'target': <function publish at 0x7fe45c626b80>, 'args': ('https://www.google.se/',), 'kwargs': {}, 'tries': 4, 'elapsed': 3.418447, 'wait': 5.113712077372433}
sleeping
called publish with url=https://www.google.se/
Traceback (most recent call last):
...