在Python中的两个标签之间获取数据

Question

<h3>
<a href="article.jsp?tp=&arnumber=16">
Granular computing based
<span class="snippet">data</span>
<span class="snippet">mining</span>
in the views of rough set and fuzzy set
</a>
</h3>

Using Python I want to get the values from the anchor tag which should be Granular computing based data mining in the views of rough set and fuzzy set

I tried using lxml

parser = etree.HTMLParser()
tree   = etree.parse(StringIO.StringIO(html), parser)                   
xpath1 = "//h3/a/child::text() | //h3/a/span/child::text()"
rawResponse = tree.xpath(xpath1)              
print rawResponse

and getting the following output

['\r\n\t\t','\r\n\t\t\t\t\t\t\t\t\tgranular computing based','data','mining','in the view of roughset and fuzzyset\r\n\t\t\t\t\t\t\]

Answer 1

You could use the text_content method:

import lxml.html as LH

html = '''<h3>
<a href="article.jsp?tp=&arnumber=16">
Granular computing based
<span class="snippet">data</span>
<span class="snippet">mining</span>
in the views of rough set and fuzzy set
</a>
</h3>'''

root = LH.fromstring(html)
for elt in root.xpath('//a'):
    print(elt.text_content())

yields

Granular computing based
data
mining
in the views of rough set and fuzzy set

or, to remove whitespace, you could use

print(' '.join(elt.text_content().split()))

to obtain

Granular computing based data mining in the views of rough set and fuzzy set

Here is another option which you might find useful:

print(' '.join([elt.strip() for elt in root.xpath('//a/descendant-or-self::text()')]))

yields

Granular computing based data  mining in the views of rough set and fuzzy set

(Note it leaves an extra space between data and mining however.)

'//a/descendant-or-self::text()' is a more generalized version of "//a/child::text() | //a/span/child::text()" . It will iterate through all children and grandchildren, etc.

Answer 2

With BeautifulSoup :

>>> from bs4 import BeautifulSoup
>>> html = (the html you posted above)
>>> soup = BeautifulSoup(html)
>>> print " ".join(soup.h3.text.split())
Granular computing based data mining in the views of rough set and fuzzy set

Explanation:

BeautifulSoup parses the HTML, making it easily accessible. soup.h3 accesses the h3 tag in the HTML.

.text , simply, gets everything from the h3 tag, excluding all the other tags such as the span s.

I use split() here to get rid of the excess whitespace and newlines, then " ".join() as the split function returns a list.