[英]Requests is unable to get page
我正在嘗試使用 Beautiful Soup 檢索此頁面:
這是我嘗試過的代碼:
import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.nasdaq.com/market-activity/stocks/msft/news-headlines")
每次我運行我的代碼時,它都會卡住並且無法檢索頁面。 但是,我收到了一次 ReadTimeout 異常( requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='www.nasdaq.com', port=443): Read timed out. (read timeout=None)
)。
對此問題的任何幫助或修復將不勝感激。
我在我的請求中包含了標題,它似乎有效。 我使用了瀏覽器發送的相同標頭,您可以使用開發人員工具找到這些標頭(如此處所示)。
import requests
headers = {
"authority": "www.nasdaq.com",
"method": "GET",
"path": "/market-activity/stocks/msft/news-headlines",
"scheme": "https",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-CA,en;q=0.9,ro-RO;q=0.8,ro;q=0.7,en-GB;q=0.6,en-US;q=0.5",
"cache-control": "max-age=0",
"dnt": "1",
"if-modified-since": "Tue, 30 Jun 2020 19:43:05 GMT",
"if-none-match": "1593546185",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"
}
page = requests.get("https://www.nasdaq.com/market-activity/stocks/msft/news-headlines", headers=headers)
而不是這樣做
import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.nasdaq.com/market-activity/stocks/msft/news-headlines")
嘗試以這種方式檢索網頁:
from urllib.request import Request, urlopen
from bs4 import BeautifulSoup
page = Request("https://www.nasdaq.com/market-activity/stocks/msft/news-headlines")
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.