Python Requests try/except continue/break loop

Question

For various reasons I need to put a Requests call inside a try/except/retry loop, rather mounting a retry condition to a requests session. Expected behaviour is that if a request has been successful, the loop breaks and the code stops. Actual behaviour though is that it repeats the loop from start to finish, with the break statement seemingly having no effect:

import traceback
import requests
import time

for i in range(0, 15):
    while True:
        try:
            headers ={
            'authority': 'www.wikipedia.org',
            'method': 'GET',
            'path': '/wikipedia.org',
            'scheme': 'https',
            'accept': '*/*',
            'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8',
            'referer': 'https://google.com',
            'sec-fetch-dest': 'empty',
            'sec-fetch-mode': 'cors',
            'sec-fetch-site': 'same-origin',
            'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36'
            }
            r = requests.get(url='https://wikipedia.org', headers=headers)
            print(i, r.status_code)
        except Exception as exc:
            time.sleep(60)
            print(traceback.format_exc())
            print('continue')
            continue
    
        print('break')
        break
    
    print('Finished')

What do I need to change to get the desired behaviour?

Answer 1

Remove while cycle. while work under for . When python come to break it "break" while cycle.

It's work for me:

import traceback
import requests
import time

for i in range(0, 15):
    try:
        headers ={
        'authority': 'www.wikipedia.org',
        'method': 'GET',
        'path': '/wikipedia.org',
        'scheme': 'https',
        'accept': '*/*',
        'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8',
        'referer': 'https://google.com',
        'sec-fetch-dest': 'empty',
        'sec-fetch-mode': 'cors',
        'sec-fetch-site': 'same-origin',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36'
        }
        r = requests.get(url='https://wikipedia.org', headers=headers)
        print(i, r.status_code)
    except Exception as exc:
        time.sleep(60)
        print(traceback.format_exc())
        print('continue')
        continue

    print('break')
    break

print('Finished')

Answer 2

try this:

import requests
from requests.exceptions import HTTPError


def get_response(url):

    for i in range(15):
        try:
            response = requests.get(url)
            response.raise_for_status()
            return response.content
        except HTTPError as http_err:
            print(f"HTTP error occurred: {http_err}")
        except Exception as err:
            print(f"Other error occurred: {err}")


url = 'http://google.com'
response = get_response(url)
print(response)
url = 'http://no_valid_address.com'
response = get_response(url)
print(response)

Answer 3

Expected behaviour is that if a request has been successful, the loop breaks and the code stops

the loop should have 15 attempts at running successfully - at which ever iteration of the loop it is successful at, it should end.

You don't need the while block. If you want to have 15 attempts or until first successful request, just do it.

import random


def random_error():
    if random.random() > 0.2:
        raise Exception


for _ in range(15):
    print('-' * 20)
    try:
        random_error()
    except Exception as exc:
        print('exception raised')
    else:
        print('exception not raised')
        break

print('Finished')