[英]How to get next page link in python beautifulsoup?
我有这个链接:
http://www.brothersoft.com/windows/categories.html
我想在div中获取项目的链接。 例:
http://www.brothersoft.com/windows/mp3_audio/midi_tools/
我试过这段代码:
import urllib
from bs4 import BeautifulSoup
url = 'http://www.brothersoft.com/windows/categories.html'
pageHtml = urllib.urlopen(url).read()
soup = BeautifulSoup(pageHtml)
sAll = [div.find('a') for div in soup.findAll('div', attrs={'class':'brLeft'})]
for i in sAll:
print "http://www.brothersoft.com"+i['href']
但我只得到输出:
http://www.brothersoft.com/windows/mp3_audio/
如何获得我需要的输出?
网址http://www.brothersoft.com/windows/mp3_audio/midi_tools/
不在标记<div class='brLeft'>
,因此如果输出为http://www.brothersoft.com/windows/mp3_audio/
,那就是正确。
如果您想获得所需的网址,请进行更改
sAll = [div.find('a') for div in soup.findAll('div', attrs={'class':'brLeft'})]
至
sAll = [div.find('a') for div in soup.findAll('div', attrs={'class':'brRight'})]
更新:
在'midi_tools'中获取信息的示例
import urllib
from bs4 import BeautifulSoup
url = 'http://www.brothersoft.com/windows/categories.html'
pageHtml = urllib.urlopen(url).read()
soup = BeautifulSoup(pageHtml)
sAll = [div.find('a') for div in soup.findAll('div', attrs={'class':'brRight'})]
for i in sAll:
suburl = "http://www.brothersoft.com"+i['href'] #which is a url like 'midi_tools'
content = urllib.urlopen(suburl).read()
anosoup = BeautifulSoup(content)
ablock = anosoup.find('table',{'id':'courseTab'})
for atr in ablock.findAll('tr',{'class':'border_bot '}):
print atr.find('dt').a.string #name
print "http://www.brothersoft.com" + atr.find('a',{'class':'tabDownload'})['href'] #link
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.