如何使用Beautiful Soup和Python提取NASDAQ网站中表格的HTML代码

Question

So basically, I want to read the HTML code for this table in the following link: https://www.nasdaq.com/symbol/aapl/revenue-eps

To do this, I have used python and Beautiful Soup.

import urllib
from bs4 import BeautifulSoup
import csv
url = urllib.urlopen("https://www.nasdaq.com/symbol/aapl/revenue-eps")
mylist = []
soup = BeautifulSoup(url,"html.parser")
my_table = soup.find('table',{'class':'ipos'})
print(my_table)

The code above is what I have attempted. When I right click and select inspect on the table, the table name I find is called 'ipos' but when I try to put it in this code, it just doesnt seem to work. The only output I get is "None"

I have tested this with another site, and it works perfectly. When i used that link and used the name of the table in that HTML code, I am able to obtain the HTML code for that table perfectly. However, that is not the case with this one. Any assistance on this would be really appreciated.

Answer 1

The table you see is inside the <iframe> . To load the content of this <iframe> you can use this script:

import requests
from bs4 import BeautifulSoup

url = "https://www.nasdaq.com/symbol/aapl/revenue-eps"
soup = BeautifulSoup(requests.get(url).text,"html.parser")

iframe_url = soup.select_one('iframe#frmMain')['src']

requests.packages.urllib3.disable_warnings()
soup = BeautifulSoup(requests.get(iframe_url, verify=False).text,"html.parser")

table = soup.select_one('table.ipos')

for tr in table.select('tr'):
    for td in tr.select('td'):
        print('{: <30}'.format(td.get_text(strip=True)), end='')
    print()

Prints:

Revenue / EPS Summary *                                     Revenue / EPS Summary *                                     
                              Revenue / EPS Summary *                                     


Fiscal Quarter                                              2019(Fiscal Year)                                           2018(Fiscal Year)                                           2017(Fiscal Year)             


December                                                                                                                
Revenue                       $84,310(m)                    $88,293(m)                    $78,351(m)                    
EPS                           4.18 (12/29/2018)             3.89 (12/30/2017)             3.36 (12/31/2016)             
Dividends                     0.73                          0.63                          0.57                          

March                                                                                                                   
Revenue                       $58,015(m)                    $61,137(m)                    $52,896(m)                    
EPS                           2.48 (3/30/2019)              2.74 (3/31/2018)              2.1 (4/1/2017)                
Dividends                     0.77                          0.73                          0.63                          

June                                                                                                                    
Revenue                       $53,809(m)                    $53,265(m)                    $45,408(m)                    
EPS                           2.2 (6/29/2019)               2.36 (6/30/2018)              1.68 (7/1/2017)               
Dividends                     0.77                          0.73                          0.63                          

September  (FYE)                                                                                                        
Revenue                                                     $62,900(m)                    $52,579(m)                    
EPS                                                         2.92 (9/29/2018)              2.07 (9/30/2017)              
Dividends                                                   0.73                          0.63                          


Totals                                                                                                                  
Revenue                       $196,134(m)                   $265,595(m)                   $229,234(m)                   
EPS                           8.86                          11.91                         9.21                          
Dividends                     2.27                          2.82                          2.46                          

Previous 3 Years   

Answer 2

The table loads inside an iframe . If you inspect the network requests made by this page then you will find a request like this:

https://fundamentals.nasdaq.com/redpage.asp?selected=AAPL&market=NASDAQ-GS&LogoPath=https%3a%2f%2fwww.nasdaq.com%2flogos%2fAAPL.GIF&coname=Apple%20Inc.

This is the url that loads the table into this page. Using the above url you can find the table.

import urllib
from bs4 import BeautifulSoup
import csv
url = urllib.urlopen("https://fundamentals.nasdaq.com/redpage.asp?selected=AAPL&market=NASDAQ-GS&LogoPath=https%3a%2f%2fwww.nasdaq.com%2flogos%2fAAPL.GIF&coname=Apple%20Inc.")
mylist = []
soup = BeautifulSoup(url,"html.parser")
my_table = soup.find('table',{'class':'ipos'})
print(my_table)