如何使用BeautifulSoup从长标签中获取部分文本

Question

I have been studying a shopping website, and I want to extract the brandname and the product name from its html code like the following:

<h1 class="product-name elim-suites">Chantecaille<span itemprop="name" >Limited Edition Protect the Lion Eye Palette</span></h1>

I tried: results = soup.findAll("h1", {"class" : "product-name elim-suites"})[0].text

and got: u'ChantecailleLimited Edition Protect the Lion Eye Palette'

As you can see, Chantecaille is the brandname, the rest is the product name, but they are now sticked to each other, any suggestion? Thank you!

Answer 1

You can use previous_sibling , which gets the previous node that has the same parent (same level in the parse tree).

Also, instead of findAll , when you are searching for a single element, use find .

item_span = soup.find("h1", {"class" : "product-name elim-suites"}).find("span")

product_name = item_span.previous_sibling
brand_name = item_span.text

print product_name
print brand_name

Output:

Chantecaille
Limited Edition Protect the Lion Eye Palette

Answer 2

You could use get_text and pass a character to separate the text or pull the text using . h1.find(text=True, recursive=False) . h1.find(text=True, recursive=False) on the h1 and pull the text from the span directly:

In [1]: h ="""<h1 class="product-name elim-suites">Chantecaille<span itemprop="name" >Limited Edition Protect the Lion Eye Palette
   ...: </span></h1>"""

In [2]: from bs4 import BeautifulSoup

In [3]: soup = BeautifulSoup(h, "html.parser")

In [4]: h1 = soup.select_one("h1.product-name.elim-suites")

In [5]: print(h1.get_text("\n"))
Chantecaille
Limited Edition Protect the Lion Eye Palette


In [6]: prod, desc = h1.find(text=True, recursive=False), h1.span.text

In [7]: print(prod, desc)
(u'Chantecaille', u'Limited Edition Protect the Lion Eye Palette\n')

Or if text could appear after the span also use find_all :

In [8]: h ="""<h1 class="product-name elim-suites">Chantecaille
<span itemprop="name" >Limited Edition Protect the Lion Eye Palette</span>other text</h1>"""


In [9]: from bs4 import BeautifulSoup

In [10]: soup = BeautifulSoup(h, "html.parser")

In [11]: h1 = soup.select_one("h1.product-name.elim-suites")

In [12]: print(h1.get_text("\n"))
Chantecaille
Limited Edition Protect the Lion Eye Palette
other text

In [13]: prod, desc = " ".join(h1.find_all(text=True, recursive=False)), h1.span.text

In [14]: 

In [14]: print(prod, desc)
(u'Chantecaille other text', u'Limited Edition Protect the Lion Eye Palette')