"尝试抓取网站时如何摆脱错误？"

Question

When I try to scrape woolworths for grocery price data, I get the following error: urllib.error.HTTPError: HTTP Error 403: Forbidden<\/em>

    session = requests.session()
url = "https://www.woolworths.com.au/shop/browse/fruit-veg"
headers = {"User-Agent":"Mozilla/5.0"}
req = session.get(url, headers = headers)
fruitAndVeg = BeautifulSoup(req.text)

Answer 1

Try using a complete user-agent such as

headers = {"User-Agent" : "Mozilla/5.0 (iPhone; CPU iPhone OS 12_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) FxiOS/96.0 Mobile/15E148 Safari/605.1.15"}