How can I end a request in a loop after a certain time in Python?

Question

My Python-program should (or can already) send requests to a list of websites to test whether they exists. Then it should save the result (whether it exists or not) in a list or afterwards in a file. So far it works well. But I always get an error message for some websites, that looks like this:

requests.exceptions.ReadTimeout: HTTPConnectionPool(host='www.oderwald.de', port=80): Read timed out. (read timeout=60)

I think the problem is that the website just doesn't send a response and keeps asking the program. So far my program looks like this:

import requests
from requests.exceptions import ConnectionError
with open("list_website.txt") as infile:
   list = [list.strip() for list in infile]


list_ge = []
list_ne = []
x=0
n=0 
g=0
i=0

for i in range(len(list)):
    try:
        request = requests.get('http://www.' + list[i] + ".de")
    except ConnectionError:
        list_ne.append(list[i])
        g=+1
        file = open('not_working.txt','a')
        file.write(list[i]+ "\n")
    else:
        list_ge.append(list[i])
        n=+1
        file2 = open('works.txt','a')
        file2.write(liste[i] + "\n")

print(list_ge)
print(list_ne)

Does anyone know how I can solve this problem? Many thanks in advance.

Answer 1

Edit:

To cause the exception to return earlier (not the default 60 seconds), change the line

request = requests.get('http://www.' + list[i] + ".de") to

request = requests.get('http://www.' + list[i] + ".de", timeout = 2) , where 2 is the amount of seconds you are willing to wait at most.

Initial post below:

The website that you are trying to reach (www.oderwald.de) does not respond in the assigned time (60 seconds in your case), so this behaviour is to be expected. Since it is just an exception, you can handle it with an except statement. See below:

try:
    request = requests.get('http://www.' + list[i] + ".de")
except requests.exceptions.ReadTimeout:
    print("Read timeout occurred")
    # The website exists but does not respond.
    # Decide to which category you assign it.

Also list() refers to an inbuilt Python function, which you are overwriting in this line: list = [list.strip() for list in infile] . Please use another variable name instead.