[英]how can i get data that is in href attribute of <a> using BeautifulSoup in python?
import requests
from bs4 import BeautifulSoup
url = 'https://www.maritimecourier.com/restaurant'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) '\
'AppleWebKit/537.36 (KHTML, like Gecko) '\
'Chrome/75.0.3770.80 Safari/537.36'}
response = requests.get(url,headers=headers)
soup = BeautifulSoup(response.text, "html.parser")
test = soup.select('.underline-body-links .sqs-block a, .underline-body-links .entry-
content a, .underline-body-links .eventlist-excerpt a, .underline-body-links
.playlist-description a, .underline-body-links .image-description a, .underline-body-
links .sqs-block a:visited, .underline-body-links .entry-content a:visited,
.underline-body-links .eventlist-excerpt a:visited, .underline-body-links .playlist-
description a:visited, .underline-body-links .image-description a:visited')
test
With this code I get this output使用此代码,我得到了这个 output
[<a href="https://www.instagram.com/breakfast_dreams/" target="_blank">Breakfast Dreams</a>,
<a href="https://www.maritimecourier.com/breakfast-dreams" target="_blank">MARITIME</a>,
<a href="https://www.instagram.com/latarantellalb/" target="_blank">La Tarantella</a>]
Now, I am trying to get the URL and the name from the a tag现在,我正在尝试从 a 标签中获取 URL 和名称
I would like to know how can I do this.我想知道我该怎么做。 So far I tried with this:
到目前为止,我试过这个:
results = []
for restaurant in soup.select('.underline-body-links .sqs-block a, .underline-body-links .entry-content a, .underline-body-links .eventlist-excerpt a, .underline-body-links .playlist-description a, .underline-body-links .image-description a, .underline-body-links .sqs-block a:visited, .underline-body-links .entry-content a:visited, .underline-body-links .eventlist-excerpt a:visited, .underline-body-links .playlist-description a:visited, .underline-body-links .image-description a:visited'):
results.append({
'title':restaurant.find('a',{'target':'_blank'}).text
})
results
But I got this但我得到了这个
'NoneType' object has no attribute 'text'
Your selection is not quiet clear and also the expected output - Main issue is that you still selected the <a>
s and try to find an <a>
in an <a>
.您的选择不是很清楚,也不是预期的 output - 主要问题是您仍然选择了
<a>
并尝试在<a>
中找到<a>
。
So your extraction part should more look like this:所以你的提取部分应该更像这样:
results.append({
'title': restaurant.text,
'url': restaurant.get('href')
})
You could also make your selection more specific:您还可以使您的选择更具体:
[{'title':a.text, 'url':a.get('href')} for a in soup.select('.sqs-block-content a')]
or with out all the internal links:或者没有所有内部链接:
[{'title':a.text, 'url':a.get('href')} for a in soup.select('.sqs-block-content a') if 'maritimecourier' not in a.get('href')]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.