使用 Beautiful Soup Python 进行网页抓取

Question

I am currently trying to web scrape a product name out of a website, however, the text is contained within a tag I have never seen before, and hence do not know how to get the text out.

<h1 class="protect" data-category="Jackets" data-ino="SS18J42" data-
rd="02/22/2018" data-rw="1SS18" itemprop="name">Gradient Puffy 
Jacket</h1>

Answer 1

You can use BeautifulSoup :

from bs4 import BeautifulSoup as soup
s = """
 <h1 class="protect" data-category="Jackets" data-ino="SS18J42" data-
rd="02/22/2018" data-rw="1SS18" itemprop="name">Gradient Puffy Jacket</h1>
"""
new_s = soup(s, 'lxml').find('h1', {'itemprop':'name'}).text

Output:

u'Gradient Puffy Jacket'

Answer 2

Adding to Ajax1234's answer. If you are searching via other html attributes:

from bs4 import BeautifulSoup
s = """
<h1 class="protect" data-category="Jackets" data-ino="SS18J42" data-
rd="02/22/2018" data-rw="1SS18" itemprop="name">Gradient Puffy Jacket</h1>
"""
soup = BeautifulSoup(s, 'html.parser')

print(soup.find('h1', {'class': 'protect'}).text)
print(soup.find('h1', {'data-category': 'Jackets'}).text)
print(soup.find('h1', {'data-ino': 'SS18J42'}).text)

etc...

Answer 3

BeautifulSoup allows you to access elements using attributes, so you could use the following approach:

from bs4 import BeautifulSoup

html = """<h1 class="protect" data-category="Jackets" data-ino="SS18J42" data-
rd="02/22/2018" data-rw="1SS18" itemprop="name">Gradient Puffy 
Jacket</h1>"""

soup = BeautifulSoup(html, "html.parser")
print soup.h1.text

Answer 4

With the help of find in beautifulsoup

soup = BeautifulSoup(html, "html.parser")
print soup.find('h1',{'class':'protect'}).text

Answer 5

We can easily find our required data using below two methods both are working well.

More ino please read documentations

First method

from bs4 import BeautifulSoup as soup
html = """
<div itemscope>  <p itemprop="a">1</p>
"""
src = soup(html, 'lxml').find('p', {'itemprop':'a'}).text
print(src)

output - 1

Second method

from bs4 import BeautifulSoup
s = """
<a class="doctor-name" itemprop="name" href="/doctors/gastroenterologists       /required-code-1689679557">Required-code-or-output</a>
"""
soup = BeautifulSoup(s, 'html.parser')

print(soup.find('a', {'class': 'doctor-name'}).text)
print(soup.find('a', {'itemprop': 'name'}).text)

output - Required-code-or-output