如何用漂亮的汤从html表中提取数据

Question

How can I extract specific data from the followed table such as decay time 91.1 ms 5?

<table bgcolor=navy cellpadding=4 cellspacing=1 border=0 align=center> 
  <tr class=hp >
    <td nowrap>E(level) (MeV)</td>
    <td nowrap>J&pi;</td><td nowrap>&Delta;(MeV)</td>
    <td nowrap>T<sub>1/2</sub></td>
    <td nowrap>Decay Modes</td>
  </tr>
  <tr class=cp>
    <td nowrap valign=top>0.0</td>
    <td nowrap valign=top>4+</td>
    <td nowrap valign=top> 18.2010</td>
    <td nowrap valign=top>91.1 ms <i>5</i>&nbsp;</td>
    <td nowrap valign=top> &epsilon; : 100.00 &#37;<br>  &epsilon;p : 55.00 &#37;<br>  &epsilon;2p : 1.10 &#37;<br>  &epsilon;&alpha; : 0.04 &#37;<br> </td>
  </tr>
</table>

Answer 1

Here's a simple code to put that table into a pandas dataframe:

from bs4 import BeautifulSoup
import pandas as pd

page = """<table cellpadding=4 cellspacing=1 border=0 align=center> 
  <tr class=hp >
    <td nowrap>E(level) (MeV)</td>
    <td nowrap>J&pi;</td>
    <td nowrap>&Delta;(MeV)</td>
    <td nowrap>T<sub>1/2</sub></td>
    <td nowrap>Decay Modes</td>
  </tr>
  <tr class=cp>
    <td nowrap valign=top>0.0</td>
    <td nowrap valign=top>4+</td>
    <td nowrap valign=top> 18.2010</td>
    <td nowrap valign=top>91.1 ms <i>5</i>&nbsp;</td>
    <td nowrap valign=top> &epsilon; : 100.00 &#37;<br>  &epsilon;p : 55.00 &#37;<br>  &epsilon;2p : 1.10 &#37;<br>  &epsilon;&alpha; : 0.04 &#37;<br> </td>
  </tr>
</table>"""

soup = BeautifulSoup(page, "html.parser")
headers = soup.find('tr', {'class':'hp'}).findAll('td')
columns = []
for header in headers:
    columns.append(header.text)

data = []
data_raw = soup.findAll('tr',{'class':'cp'})
for row in data_raw:
    items = []
    for element in row.findAll('td'):
        items.append(element.text)
    data.append(items)

df = pd.DataFrame(data, columns=columns)

print(df['T1/2'])

Output is:

0    91.1 ms 5 
Name: T1/2, dtype: object

If what you have in Decay Modes are multiple rows you may have to add additional code to detect that (they are separated by <br> ), or if you can, correct the HTML to have different rows within different row tags and the header in a header tag

Answer 2

您可以使用get_element_by_tag_name获取表并遍历每个内部标签并获取必要的数据。

Answer 3

Assuming you have the markup already in a string. You have to find the elements by class (.cp), then you have to find by tag (td), and you can get the value of each found element using .text atribute, so use the following code:

import re
from bs4 import BeautifulSoup

html_doc = """<table bgcolor=navy cellpadding=4 cellspacing=1 border=0 align=center> 
  <tr class=hp >
    <td nowrap>E(level) (MeV)</td>
    <td nowrap>J&pi;</td><td nowrap>&Delta;(MeV)</td>
    <td nowrap>T<sub>1/2</sub></td>
    <td nowrap>Decay Modes</td>
  </tr>
  <tr class=cp>
    <td nowrap valign=top>0.0</td>
    <td nowrap valign=top>4+</td>
    <td nowrap valign=top> 18.2010</td>
    <td nowrap valign=top>91.1 ms <i>5</i>&nbsp;</td>
    <td nowrap valign=top> &epsilon; : 100.00 &#37;<br>  &epsilon;p : 55.00 &#37;<br>  &epsilon;2p : 1.10 &#37;<br>  &epsilon;&alpha; : 0.04 &#37;<br> </td>
  </tr>
</table>"""

soup = BeautifulSoup(html_doc, 'html.parser')
elements = soup.find_all(class_=re.compile("cp"))

for e in elements[0].find_all('td'):
    # the e.text contains the value of each td elements in your table
    print(e.text)

Answer 4

Ususally if I see a <table> tag, using pandas .read_html() is my first thing to try. It'll retunr a list of dataframes. It's then just a matter of selecting the dataframe and manipulating the dataframe to get it the way you want, or to pull the data you need:

import pandas as pd


html = '''<table bgcolor=navy cellpadding=4 cellspacing=1 border=0 align=center> 
  <tr class=hp >
    <td nowrap>E(level) (MeV)</td>
    <td nowrap>J&pi;</td><td nowrap>&Delta;(MeV)</td>
    <td nowrap>T<sub>1/2</sub></td>
    <td nowrap>Decay Modes</td>
  </tr>
  <tr class=cp>
    <td nowrap valign=top>0.0</td>
    <td nowrap valign=top>4+</td>
    <td nowrap valign=top> 18.2010</td>
    <td nowrap valign=top>91.1 ms <i>5</i>&nbsp;</td>
    <td nowrap valign=top> &epsilon; : 100.00 &#37;<br>  &epsilon;p : 55.00 &#37;<br>  &epsilon;2p : 1.10 &#37;<br>  &epsilon;&alpha; : 0.04 &#37;<br> </td>
  </tr>
</table>'''

tables = pd.read_html(html)
df = tables[0]
df.columns = df.iloc[0,:]
df = df.iloc[1:,:]

Output:

print(df.loc[1,'T1/2'])
91.1 ms 5