美丽的汤和Python，嵌套元素

Question

I am attempting to scrape nested elements via BeautifulSoup and I have been pulling my hair out for a couple days now. I am, by far, a novice -- so I hope the simplicity of this question does not offend anyone. Still, any help in any capacity would be greatly appreciated.

Here is the html I'm attempting to scrape.

        <div id="specs" class="pane">
           <div class="col">
              <ul class="list">
                 <li>
                    <ul>
                       <li><b>width</b>2</li>
                       <li><b>length</b>1</li>
                       <li><b>color</b>blue</li>
                       <li><b>metal</b>steel</li>
                    </ul>
                 </li>
              </ul>
           </div>
        </div>

And in a perfect world, here is my result...

width, 2
length, 1
color, blue
metal, steel

While I've come close, I know now this can't be the answer...yet, at the same time, I can't seem to loop through the li elements.

div = div.find("div", {"id":"specifications"})
result = [i for i in div.find('li')]

If anyone can just push aa beginner in the right direction, it would be greatly greatly appreciation, and thank you in advance for any insight!

Answer 1

You can use CSS selector via select() to find the target b elements, for example :

from bs4 import BeautifulSoup
raw = '''<div id="specs" class="pane">
           <div class="col">
              <ul class="list">
                 <li>
                    <ul>
                       <li><b>width</b>2</li>
                       <li><b>length</b>1</li>
                       <li><b>color</b>blue</li>
                       <li><b>metal</b>steel</li>
                    </ul>
                 </li>
              </ul>
           </div>
        </div>'''
soup = BeautifulSoup(raw, "lxml")

result = soup.select("div#specs b")    
for r in result:
    print r.get_text(), r.next_sibling

output :

width 2
length 1
color blue
metal steel

---

The following is a pure lxml.html alternative for comparison (since OP seems interested in lxml , judging from his comment below). The output is exactly the same as BS snippet above.

from lxml import html
raw = '''assume the same XML as in the previous snippet'''
root = html.fromstring(raw)

result = root.cssselect("div#specs b")
for b in result:
    print b.text, b.tail

lxml supports both XPath (via xpath() ) and CSS selector (via cssselect() ), and lxml is fast .