如何使用 Python 從 javascript Piechart 圖中抓取節點文本

Question

如何使用這樣的 Python 從 javascript Piechart 圖中抓取節點？

https://www.dice.com/skills/javascript

餅圖

提示：我希望從圖中抓取的文本代表圖節點，而不是普通文本。

Answer 1

實際上頁面是通過JavaScript呈現的，因此我們可以使用selenium或使用requests和bs4因為所需的輸出位於script標記中，可以使用regex捕獲

from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.firefox.options import Options
from bs4 import BeautifulSoup

options = Options()
options.add_argument('--headless')

driver = webdriver.Firefox(options=options)
driver.get(
    'https://www.dice.com/skills/javascript')

soup = BeautifulSoup(driver.page_source, 'html.parser')

for item in soup.findAll("div", {'class': 'node'}):
    print(item.text)
driver.quit()

輸出：

JavaScript
CSS
HTML5
jQuery
jQuery UI
AngularJS
Bootstrap
jQuery
jQuery UI
CSS
HTML5
AngularJS
Ajax
jQuery UI
jQuery
Aptana
Zend Studio
Ajax
CSS
AngularJS
jQuery
HTML5
HTML
Bootstrap
CSS
Node.js
React.js
MongoDB
AngularJS
Express.js
NoSQL

更新：

import requests
from bs4 import BeautifulSoup


r = requests.get("https://www.dice.com/skills/javascript")

soup = BeautifulSoup(r.text, 'html.parser')

script = soup.findAll("script")[8].get_text("\t", strip=True)

start = script.find("{")
end = script.find(";")

print(script[start:end])

輸出：

{"name":"JavaScript","children":[{"name":"CSS","children":[{"name":"HTML5"},{"name":"jQuery"},{"name":"jQuery UI"},{"name":"AngularJS"},{"name":"Bootstrap"}]},{"name":"jQuery","children":[{"name":"jQuery UI"},{"name":"CSS"},{"name":"HTML5"},{"name":"AngularJS"},{"name":"Ajax"}]},{"name":"jQuery UI","children":[{"name":"jQuery"},{"name":"Aptana"},{"name":"Zend Studio"},{"name":"Ajax"},{"name":"CSS"}]},{"name":"AngularJS","children":[{"name":"jQuery"},{"name":"HTML5"},{"name":"HTML"},{"name":"Bootstrap"},{"name":"CSS"}]},{"name":"Node.js","children":[{"name":"React.js"},{"name":"MongoDB"},{"name":"AngularJS"},{"name":"Express.js"},{"name":"NoSQL"}]}]}