[英]extract data from html content
I want to download some html pages and extract informations, each HTML page has this table tag
: 我想下载一些html页面并提取信息,每个HTML页面都有这个table tag
:
<table class="sobi2Details" style='background-image: url(http://www.imd.ir/components/com_sobi2/images/backgrounds/grey.gif);border-style: solid; border-color: #808080' >
<tr>
<td><h1>Dr Jhon Doe</h1></td>
</tr>
<tr>
<td></td>
</tr>
<tr>
<td></td>
</tr>
<tr>
<td>
<div id="sobi2outer">
<br/>
<span id="sobi2Details_field_name" ><span id="sobi2Listing_field_name_label">name:</span>Jhon</span><br/>
<span id="sobi2Details_field_family" ><span id="sobi2Listing_field_family_label">family:</span> Doe</span><br/>
<span id="sobi2Details_field_tel1" ><span id="sobi2Listing_field_tel1_label">tel:</span> 33727464</span><br/>
</div>
</td>
</tr>
</table>
I want to access name ( Jhone
) ,family ( Doe
) and tel( 33727464
),I've used beausiful soup to access these span tags by id: 我想访问名称( Jhone
),家庭( Doe
)和tel( 33727464
),我使用beausiful汤通过id访问这些span标签:
name=soup.find(id="sobi2Details_field_name").__str__()
family=soup.find(id="sobi2Details_field_family").__str__()
tel=soup.find(id="sobi2Details_field_tel1").__str__()
but I don't know how to extract data into these tags.I tryed to use children
and content
attributes,but when I use theme as a tag
It returns None
: 但我不知道如何将数据提取到这些tag
。我试图使用children
和content
属性,但当我使用主题作为tag
它返回None
:
name=soup.find(id="sobi2Details_field_name")
for child in name.children:
#process content inside
but I get this error: 但我得到这个错误:
'NoneType' object has no attribute 'children'
while when I use str () on it,it is not None
!! 当我在它上面使用str ()时,它不是None
!! any Idea? 任何想法?
Edit:My final solution 编辑:我的最终解决方案
soup = BeautifulSoup(page,from_encoding="utf-8")
name_span=soup.find(id="sobi2Details_field_name").__str__()
name=name_span.split(':')[-1]
result = re.sub('</span>', '',name)
I found a couple of ways to do it. 我发现了几种方法。
from bs4 import BeautifulSoup
soup = BeautifulSoup(open(path_to_html_file))
name_span = soup.find(id="sobi2Details_field_name")
# First way: split text over ':'
# This only works because there's always a ':' before the target field
name = name_span.text.split(':')[1]
# Second way: iterate over the span strings
# The element you look for is always the last one
name = list(name_span.strings)[-1]
# Third way: iterate over 'next' elements
name = name_span.next.next.next # you can create a function to do that, it looks ugly :)
Tell me if it helps. 告诉我它是否有帮助。
If you are familiar with xpath use lxml with etree instead: 如果您熟悉xpath,请使用带有etree的lxml:
import urllib2
from lxml import etree
opener = urllib2.build_opener()
root = etree.HTML(opener.open("myUrl").read())
print root.xpath("//span[@id='sobi2Details_field_name']/text()")[0]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.