[英]Python script using beautifulSoup to scrape webpage
我试图使用BeautifulSoup刮掉下一页的内容,
<div data-referrer="pagelet_123" id="pagelet_123">
<div id="1" class="p1">
<div class="uiHeader uiHeaderTopAndBottomBorder uiHeaderSection">
<div class="clearfix uiHeaderTop">
<div>
<h4 class="uiHeaderTitle">info - 1</h4>
</div></div></div><div class="phs">
<table class="uicontenttable">
<tbody>
<tr>
<th class="label">Other</th>
<td class="data"><div id="ua94ty_3" class="uiCollapsedList uiCollapsedListHidden uiCollapsedListNoSeparate pagesListData">
<span class="visible">
<a href="http://abc.com/Federer">info-2</a>,
<a href="http://abc.com/pages/Ian-Wright-Out-of-Bounds/117602014955747">info-3</a>,
<a href="http://abc.com/JuniperNetworks">info-4</a>,
<a href="http://abc.com/pages/Join-Diaspora/118635234836351">info-5</a>
</span>
</div>
</td>
<td class="rightCol">
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div data-referrer="pagelet_ent" id="pagelet_ent">
<div id="2" class="section2">
<div class="uiHeader uiHeaderTopAndBottomBorder uiHeaderSection">
<div class="clearfix uiHeaderTop">
<div>
<h4 class="uiHeaderTitle">info-6</h4>
</div></div></div>
<div class="phs"><table class="uiInfoTable mtm profileInfoTable">
<tbody>
<tr>
<th class="label">info - 7</th><td class="data">
<div class="mediaRowWrapper ">
<ul class="uiList uiListHorizontal clearfix pbl mediaRow">
<li class="uiListItem uiListHorizontalItemBorder uiListHorizontalItem">
<a href="URL - 1">
<div class="mediaPortrait">
<div style="height: 75px; width: 75px;" class="fbProfileScalableThumb photo">
<img width="87.00090480941" style="margin: -6px 0 0 -6px;" title="Hans Zimmer" alt="" src="http://profile.ak.fbcdn.net/hprofile-ak-snc4/203614_7170054127_6578457_s.jpg" class="img"></div><div class="mediaPageName">info - 8</div></div></a></li><li class="pls uiListItem uiListHorizontalItemBorder uiListHorizontalItem">
<a href="URL - 2">
<div class="mediaPortrait"><div style="height: 75px; width: 75px;" class="fbProfileScalableThumb photo"><img width="87.00090480941" style="margin: -6px 0 0 -6px;" title="test" alt="" src="http://external.ak.fbcdn.net/safe_image.php?d=AQCVRllyopjA_z5F&w=100&h=300&url=http%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F5%2F59%2F-2.jpg&fallback=hub_music&prefix=s" class="img"></div><div class="mediaPageName">test</div></div></a>
</div>
<div class="mediaPageName">info - 8
</div>
</div>
</a>
此页面包含多个嵌套的div和表。 需要帮助使用BeautifulSoup只解析信息 - 1信息-2 ...信息-6和URL - 1和URL -2。
我阅读了BeautifulSoup的文档,它没什么用处。 还请推荐一些BeautifulSoup参考文档,用于解析复杂网页。
谢谢你的帮助,谢谢!
SAT
他们的文档不符合您的目的?
http://www.crummy.com/software/BeautifulSoup/documentation.html
它看起来像你想要的东西:
from BeautifulSoup import BeautifulSoup
import re
soup = BeautifulSoup(theXMLAsAString)
results = soup.findAll(re.compile('info - [1-6]'))
for r in results:
myurl = r.parent.href
该代码未经过测试,但是如何使用BeautifulSoup的一般概念。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.