Python：使用BeautifulSoup从HTML提取分隔文本

Question

I have the following HTML repeated several times on a page (please do not judge):

 <div class="container">
    <div class="image">
      <a href="#" title="#" class="#">
        <img src="img.jpg" alt="#" class="#">
      </a>
    </div>
    <div class="text">
        <a href="#">
          <h4 class="h4-class">{TITLE}</h4>
        {SOME TEXT 1}<br />
        <h5><img src="img.jpg" alt="#" /> {SOME TEXT 2}</h5>
        {SOME TEXT 3}      </a>
    </div>
  </div>

I would like to extract {TITLE} , {SOME TEXT 1} , {SOME TEXT 2} and {SOME TEXT 3}

My code is as follows:

from BeautifulSoup import BeautifulSoup as bs
import urllib2
html = urllib2.urlopen('text')
soup = bs(html)
divs = soup.findAll("div", { "class" : "text" })

for div in divs:
    inner_text = div.text
    strings = inner_text.split("\n")
    print strings[0] ## I want this to print just {TITLE}

On printing it out, it prints one line connecting all the values eg

{TITLE}{SOME TEXT 1}{SOME TEXT 2}{SOME TEXT 3}

Is there anyway around this? What have I missed?

Answer 1

You can prettify ( see documentation here ) the div content first and then manipulate each line as needed. This will work if the divs with class name text have same structure.

Code (Python 2):

from BeautifulSoup import BeautifulSoup as bs

html = '''
<div class="container">
    <div class="image">
      <a href="#" title="#" class="#">
        <img src="img.jpg" alt="#" class="#">
      </a>
    </div>
    <div class="text">
        <a href="#">
          <h4 class="h4-class">{TITLE}</h4>
        {SOME TEXT 1}<br />
        <h5><img src="img.jpg" alt="#" /> {SOME TEXT 2}</h5>
        {SOME TEXT 3}      </a>
    </div>
  </div>
'''
soup = bs(html)
divs = soup.findAll("div",{"class":"text"})
for div in divs:
    pretty_div = div.prettify()
    content_list = pretty_div.split("\n")
    content_list = [s.strip() for s in content_list]
    print content_list[3]
    print content_list[5]
    print content_list[9]
    print content_list[11]

Output:

{TITLE}
{SOME TEXT 1}
{SOME TEXT 2}
{SOME TEXT 3}