[英]Find href using Beautiful Soup - Python
I'm trying to extract the first link from a page search, using beautiful soup, but it can't find the link for some reason.我正在尝试使用漂亮的汤从页面搜索中提取第一个链接,但由于某种原因找不到链接。
from requests import get
from bs4 import BeautifulSoup
import requests
band = "it's my life bon jovi"
url = f'https://www.letras.mus.br/?q={band}'
res = requests.get(url)
soup = BeautifulSoup(res.content, 'html.parser')
linkurl = soup.find_all("div", class_="wrapper")
for urls in linkurl:
print(urls.get('href'))
#print(soup.a['href']) -- return /
#print(soup.a['data-ctorig]) -- return nothing
I would like to get the link of the data-ctorig or the href , does this link have a script that is preventing me from looking for this information, or is it a problem with my code?我想获取data-ctorig或href的链接,此链接是否有阻止我查找此信息的脚本,还是我的代码有问题?
The website uses google programmable search engine (CSE) to return cached results.该网站使用谷歌可编程搜索引擎(CSE)返回缓存结果。 This required JavaScript to run in a browser which doesn't happen with requests.
这需要 JavaScript 在浏览器中运行,而请求不会发生。
It is far easier to use selenium and a more targeted css selector list to retrieve results.使用 selenium 和更有针对性的 css 选择器列表来检索结果要容易得多。
While the wait doesn't seem to be needed in this case I have added it for good measure.虽然在这种情况下似乎不需要等待,但我已经添加了它。
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
band = "it's my life bon jovi"
url = f'https://www.letras.mus.br/?q={band}'
d = webdriver.Chrome()
d.get(url)
links = WebDriverWait(d,10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".gsc-thumbnail-inside .gs-title[target]")))
links = [link.get_attribute('href') for link in links]
print(links[0])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.