retry with python requests when status_code = 200

Question

The API I'm sending requests to has a bit of an unusual format for its responses

1. It always returns status_code = 200

2. There's an additional error key inside the returned json that details the actual status of the response:

   2.1. error = 0 means it successfully completes

   2.2. error != 0 means something went wrong

I'm trying use the Retry class in urlib3 , but so far I understand it only uses the status_code from the response, not its actual content.

Are there any other options?

Answer 1

I think you're going to have to use a 'hybrid' approach here and manually do a retry after analysis.

If I'm hearing you right, the API always returns status_code 200 even if your request is bad. So we need to handle two cases, and write our own Retry handler(s). You might try something like this approach, which also takes into account the number of requests you're making to determine if there's a repeated error case.

import requests
import time
import json


r = None
j = None

attempts = 0

while (attempts < 5):
    # Make request, make sure you have the right data= for payload or json= for
    # JSON payloads in your request.  And repolace `get` with the proper method
    # such as POST or PUT if it's not a GET.
    attempts += 1
    r = requests.get('http://api.example.com/', data=...)
    if r.status_code != 200:
        time.sleep(5)  # Wait between requests for 5 seconds
        continue  # Go back to the beginning of the loop
    else:
        # Decode the bytes in the reply content, then parse it as JSON.
        j = json.loads(r.content.decode('UTF-8'))
        # Check what the output of the 'error' key in the JSON is
        if j['error'] != 0:
            # If there's an error code, then restart the loop and request
            # after a 5 second pause again.
            time.sleep(5)
            continue
        else:
            # If there is no error, simply break the loop, and go on to
            # processing code after it.
            break

if attempts >= 5:
    raise RuntimeError("Consistently invalid or error case returned from API.")


# If there's no error then...
# Do something with the output of `j` which is now a dict containing the parsed
# JSON data.

...