Dynamic parameters with Python's retry decorator

Question

I'm currently using the python retrying package which contains the @retry decorator that has a number of optional parameters. I have these parameters set correctly for our production environment with long enough wait times in between retries (below it's set to 2000 milliseconds) but I would like to set these values differently for unit testing purposes so that execution is very quick.

For example here the wait_fixed time is set to 2000 milliseconds for production but for my unit test that calls some_function() I'd like to override the wait_fixed parameter to be 1 millisecond so it executes very quickly.

@retry(stop_max_attempt_number=3, wait_fixed=2000)
def some_function(self):
    return True

The problem I'm running into is that the decorator is interpreted when the function is defined so as of yet I have not found a way to override the wait_fixed parameter from my unit tests.

Answer 1

main.py

import settings
from retrying import retry


@retry(stop_max_attempt_number=settings.STOP_MAX_ATTEMPT_NUMBER, 
       wait_fixed=settings.WAIT_FIXED)
def some_function(self):
    return True

settings.py

import json


with open('config.json') as jsonf:
    config = json.loads(jsonf.read())

WAIT_FIXED=int(config['WAIT_FIXED'])
STOP_MAX_ATTEMPT_NUMBER=int(config['STOP_MAX_ATTEMPT_NUMBER'])

config.json

{
    "STOP_MAX_ATTEMPT_NUMBER" : "3",
    "WAIT_FIXED" : "2000"
}

Have your test runner put a suitable config.json in place.

Answer 2

I recently come across a problem which having a def inside a def resolved the issue.

---

I need to use the retrying decorator on get_url function but need to pass configurable value for stop_max_attempt_number . There is no way I can come up to have get_url , along with other functions at the same indent.

To solve the issue, I have the get_url function defined inside get_analytic function. For illustration:

def get_analytics(conf: Configuration, assets: list, cache: dict) -> list:
    STOP_MAX_ATTEMPT_NUMBER = int(conf.num_retry)
    WAIT_FIXED = int(conf.retry_timeout)
    @retry(stop_max_attempt_number=STOP_MAX_ATTEMPT_NUMBER, wait_fixed=WAIT_FIXED)
    def get_url(disable_https: bool, url: str, user: str, password: str) -> Response:

Answer 3

I needed to be able to set the retry parameters dynamically when the function is called, the below worked well for me:

import random
from retrying import retry


class MyClass(object):

    def try_stuff(self, string, attempts, wait):
        self.do_something_with_retry = retry(stop_max_attempt_number=attempts,
                                             wait_fixed=wait)(self.do_something_unreliable)
        return self.do_something_with_retry(string)

    def do_something_unreliable(self, string):
        print(string)
        if random.randint(0, 10) > 1:
            raise IOError("Broken sauce, everything is hosed!!!111one")
        else:
            return "Awesome sauce!"

instance = MyClass()

print instance.try_stuff('test', 10, 2000)