我在 beautifulsoup 中找不到桌子

Question

所以我對這些東西真的很陌生，所以這可能很愚蠢。 但我似乎無法弄清楚為什么這行非常基本的代碼找不到任何表格......還試圖找到一個表格來基本上獲取每一行。 網址： https : //uwflow.com/course/cs136

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = "https://uwflow.com/course/cs136"

# opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

# html parser
page_soup = soup(page_html, "html.parser")

    enter code here

table = page_soup.findAll('table')
print(table)

Answer 1

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
import pandas as pd

options = Options()
options.add_argument('--headless')

driver = webdriver.Firefox(options=options)
driver.get("https://uwflow.com/course/cs136")

df = pd.read_html(driver.page_source)[0]

df.to_csv('out.csv', index=False)

driver.quit()

輸出： 在線查看

我已經選擇了Winter 2020 ，如果您想要Spring 2020所以您需要將[0]更改為[1]

Answer 2

數據存儲在<script>標簽內的頁面內。 如果您想要沒有selenium解決方案，您可以使用re和json模塊來解析數據。

例如：

import re
import json
import requests

url = 'https://uwflow.com/course/cs136'

txt = requests.get(url).text

data = json.loads(re.findall(r'window.pageData.courseObj = (\{.*?});', txt)[0])

# print(json.dumps(data, indent=4)) # <-- uncomment this to see all data

print(data['code'] + ' - ' + data['name'])
print(data['description'])

print('{:<10} {:<10} {:<10}'.format('Class', 'Enrolled', 'Campus'))
for section in data['sections']:
    print('{:<10} {:<10} {:<10}'.format(section['class_num'],
        str(section['enrollment_total']) + '/' + str(section['enrollment_capacity']),
        section['campus']))

印刷：

CS 136 - Elementary Algorithm Design and Data Abstraction
This course builds on the techniques and patterns learned in CS 135 while making the transition to use an imperative language. It introduces the design and analysis of algorithms, the management of information, and the programming mechanisms and methodologies required in implementations. Topics discussed include iterative and recursive sorting algorithms; lists, stacks, queues, trees, and their application; abstract data types and their implementations.
Class      Enrolled   Campus    
6214       90/90      UW U      
6011       59/65      UW U      
5914       46/90      UW U      
6004       90/90      UW U      
6048       90/90      UW U      
6109       90/90      UW U      
6215       87/90      UW U      
6260       90/90      UW U      
6261       67/90      UW U      
6005       64/65      UW U      

... and so on.