简体   繁体   中英

Web Scraping a table using python beautiful soup

I am trying to get the information from a webpage table.

https://grs.icarda.org/accessions/?IG=46860

I want 'collecting information' from the second table. But there are no tags or ids to easily parse the data.

table = soup.find('td', colspan='9')
table_data = soup.find('td', {'width':'150px', 'height':'26px'})

Here's the HTML

 <tbody><tr><td colspan="9" style=" background-color: #FFFFFF; font-weight: bold; height:33px;">Collecting information</td></tr> <tr style="background-color:#d7d4d4; height:26px;"><td style="vertical-align:middle;">Mission code:</td><td style="width:5px;"></td><td style="vertical-align:middle;">TUR79-2</td> <td width="20"></td></tr> <tr><td style="width:250px; height:26px;">Mission title:</td><td width="5"></td><td>MJ Metzger, S.Jana (USDA report)</td></tr> <tr style="background-color:#d7d4d4; height:26px;"><td style="vertical-align:middle;">Country:</td><td style="width:5px;"></td><td colspan="2" style="vertical-align:middle;"><img style="vertical-align:middle; width:24px; height:24px;" src="../images/flags/TUR.png"><span style="vertical-align:middle;"> &nbsp;Turkey</span></td></tr><tr> </tr><tr><td style="width:150px; height:26px;">Site Code:</td><td width="5"></td><td colspan="2">TUR79-2:12</td></tr><tr style="background-color:#d7d4d4; height:26px;"><td style="width:150px; height:26px;">Collectors:</td><td width="5"></td><td colspan="2">JA Hoffmann - M. Kanbertay - MJ Metzger - H. Sencer</td></tr><tr style=" height:26px;"><td style="width:150px; height:26px;">Collect Date:</td><td width="5"></td><td colspan="2">1979/08/09</td></tr> <tr style="background-color:#d7d4d4;"><td style=" width:150px; height:26px;">Collector's number:</td><td width="5"></td><td colspan="2">79TK012-057</td></tr><tr style=" height:26px;"><td style="width:150px; height:26px;">Admin 1:</td><td width="5"></td><td colspan="2">Malatya Province</td></tr><tr style=" background-color:#d7d4d4;height:26px;"><td style="width:150px; height:26px;">Admin 2:</td><td width="5"></td><td colspan="2"></td></tr><tr style="height:26px;"><td style="width:150px; height:26px;">Collecting site:</td><td width="5"></td><td colspan="2">5 km S of Darende</td></tr> </tbody>

You could use pandas which has read_html(), which returns a list of dataframes for each table on the page. That table is the 3rd one (index 2) so this code could help you. After getting the table I put the two columns into a dict for you:

import pandas as pd

df = pd.read_html('https://grs.icarda.org/accessions/?IG=46860')[2] #3rd table

col1 = df[0]
col2 = df[2]

zipped = zip(col1,col2)

output = {}
for x,y in zipped:
    output[x] = y

print(output)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM