如何在 try except 场景中捕获 HTTP Error 429“Retry-After”时间？

Question

我创建了一个 python 脚本，它使用 urllib 从 met.no 获取信息。 它是这样的：

from urllib.request import Request, urlopen
from urllib.error import HTTPError
from urllib.error import URLError
import json
import datetime, time
from datetime import datetime
from dateutil import tz

def get_info_from_yr():
    
    url = 'https://api.met.no/weatherapi/locationforecast/2.0/compact?lat70&lon=j8'
    
    headers = {
        'User-Agent': 'Python User Agent 1.0',
        'From': 'what@ever.com'
        }

    request = Request(
        url=url,
        headers= headers, 
        method='GET')
    
      
    try:
        result = urlopen(request, timeout=5)
        
    except HTTPError as http_error:
        if http_error.status == 429:
            print ("We must wait!")
            time.sleep(int(result.headers["Retry-After"]))
        else:
            print(f"Could not reach {request.full_url} due to an error: {http_error}") 
    
    except URLError as url_error:
        print (f" A URL Error occured: {url_error}")
    
    except Exception as generic_http_error:  
        print (f" A URL exception occured: {generic_http_error}")
        
    else:
        data = json.load(result)
        #print (result.status)
        http_expiry_date = convert_date_from_gmt(result.headers['Expires'])
        
        yr_data = {
                "expiry_date": http_expiry_date,
                "temperature": data["properties"]["timeseries"][0]["data"]["instant"]["details"]["air_temperature"]
            }
        
        try:
            with open("yr_data.json", "w") as wj:
                wj.write(json.dumps(yr_data, indent=2))
                
        except IOError as io_error:
            print(f" There was an IOErorr: {io_error}")
        
        except Exception as generic_io_error:  
            print (f"An IO exception occured: {generic_io_error}")

但是如果我引发 HTTP 429 错误，我会得到这个错误： time.sleep(int(result.headers["Retry-After"])) UnboundLocalError: local variable 'result' referenced before assignment

结果变量似乎没有出现在 except 块中。 那么我真的不明白如何捕捉“重试之后” header

我尝试以多种不同的方式捕捉“重试后”header，但没有人坚持

有关如何解决此问题的任何提示？

我当然可以设置一个 static 计时器，但这有它自己的缺点。

谢谢

Answer 1

根据文档，您可以从HTTPError访问响应，其中包含响应的标头。

运行您提供的 URL，我可以获得 400 响应，将您的except HTTPError as http_error块：

except HTTPError as http_error:
    if http_error.status == 429:
        print("We must wait!")
        time.sleep(int(result.headers["Retry-After"]))
    else:
        print(http_error.headers.items())
        retry_after = next((x for x in http_error.headers.items() if x[0] == "X-Backend-Host"), ("dummy", "Retry After not found, use some default value here"))[1]
        print(retry_after)
        
        print(f"Could not reach {request.full_url} due to an error: {http_error}")

回报：

[ilias@yellowhat-37 PYTHON_LEARNING] > ./requests_403.py 
[('Server', 'nginx/1.18.0 (Ubuntu)'), ('Date', 'Tue, 31 Jan 2023 20:58:58 GMT'), ('Content-Type', 'text/plain;charset=UTF-8'), ('Content-Length', '122'), ('Connection', 'close'), ('X-ErrorClass', 'Parameter'), ('Expires', '0'), ('Cache-Control', 'private, max-age=0, no-cache, no-store, must-revalidate'), ('X-Backend-Host', 'b_157_249_72_213_loc'), ('X-Varnish', '59429260'), ('Age', '0'), ('Via', '1.1 varnish (Varnish/7.0)')]
b_157_249_72_213_loc
Could not reach https://api.met.no/weatherapi/locationforecast/2.0/compact?lat=70&lon=k8 due to an error: HTTP Error 400: Bad Request
[ilias@yellowhat-37 PYTHON_LEARNING] > 

它确实可以提供响应的标题。

（您会注意到我已经用X-Backend-Host替换了 header retry-after的重试以进行概念验证。）