堆棧內存溢出
  簡體   English   中英

如何從網站上抓取股票表。 我認為 class 或標簽是一個問題，但我不知道

[英]How to scrape a stock table from a website. I think class or tag is a problem but I can't figure out

 原文  2020-12-21 05:00:42  0   1    python/ web-scraping/ beautifulsoup/ python-requests

問題描述

如何提取股票表數據？ 這是我嘗試過的代碼。

import requests
from bs4 import BeautifulSoup

url ="https://finance.daum.net/domestic/market_cap"
res=requests.get(url)
res.raise_for_status()
soup =BeautifulSoup(res.text, "lxml")

data_rows=soup.find("div",attrs={"class":"box_contents"}).find("tbody").find_all("tr")
for row in data_rows:
 columns = row.find_all("td")
 data = [column.get_text().strip() for column in columns]
 print(data)

1 個解決方案

解決方案1
 1  2020-12-21 06:04:28

我不懂韓語，但我做過測試，應該沒問題。

該信息來自其他Api Urls ，您需要在F12 Network上找到它。

Api Url

https://finance.daum.net/api/trend/market_capitalization

參數示例

page: 1
perPage: 30
fieldName: marketCap
order: desc
market: KOSPI
pagination: true

result url  = 'https://finance.daum.net/api/trend/market_capitalization?page=1&perPage=30&fieldName=marketCap&order=desc&market=KOSPI&pagination=true'

這應該是你想要的。

import requests
url = 'https://finance.daum.net/api/trend/market_capitalization?page=1&perPage=30&fieldName=marketCap&order=desc&market=KOSPI&pagination=true'
headers = {
    'referer': 'https://finance.daum.net/domestic/market_cap',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36'}
res = requests.get(url=url, headers=headers)
print(res.json())

Output

{'data': [{'rank': 1, 'name': '삼성전자', 'symbolCode': 'A005930', 'code': 'KR7005930003', 'tradePrice': 72900.0, 'change': 'FALL', 'changePrice': 100.0, 'changeRate': -0.001369863, 'marketCap': 435197147895000.0, 'listedShareCount': 5969782550, 'foreignRatio': '55.950'}, {'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 115500.0, 'change': 'FALL', 'changePrice': 3000.0, 'changeRate': -0.0253164557, 'marketCap': 84084273157500.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}, {'rank': 3, 'name': 'LG화학', 'symbolCode': 'A051910', 'code': 'KR7051910008', 'tradePrice': 816000.0, 'change': 'RISE', 'changePrice': 2000.0, 'changeRate': 0.0024570025, 'marketCap': 57603351888000.0, 'listedShareCount': 70592343, 'foreignRatio': '42.360'}, {'rank': 4, 'name': '삼성전자우', 'symbolCode': 'A005935', 'code': 'KR7005931001', 'tradePrice': 68500.0, 'change': 'FALL', 'changePrice': 100.0, 'changeRate': -0.0014577259, 'marketCap': 56367738950000.0, 'listedShareCount': 822886700, 'foreignRatio': '82.480'}, {'rank': 5, 'name': '삼성바이오로직스', 'symbolCode': 'A207940', 'code': 'KR7207940008', 'tradePrice': 822000.0, 'change': 'RISE', 'changePrice': 20000.0, 'changeRate': 0.0249376559, 'marketCap': 54387630000000.0, 'listedShareCount': 66165000, 'foreignRatio': '10.270'}, {'rank': 6, 'name': '셀트리온', 'symbolCode': 'A068270', 'code': 'KR7068270008', 'tradePrice': 367000.0, 'change': 'RISE', 'changePrice': 11000.0, 'changeRate': 0.0308988764, 'marketCap': 49544194435000.0, 'listedShareCount': 134997805, 'foreignRatio': '20.810'}, {'rank': 7, 'name': 'NAVER', 'symbolCode': 'A035420', 'code': 'KR7035420009', 'tradePrice': 285000.0, 'change': 'RISE', 'changePrice': 2500.0, 'changeRate': 0.0088495575, 'marketCap': 46815067575000.0, 'listedShareCount': 164263395, 'foreignRatio': '55.700'}, {'rank': 8, 'name': '현대차', 'symbolCode': 'A005380', 'code': 'KR7005380001', 'tradePrice': 187000.0, 'change': 'FALL', 'changePrice': 2000.0, 'changeRate': -0.0105820106, 'marketCap': 39955950969000.0, 'listedShareCount': 213668187, 'foreignRatio': '30.570'}, {'rank': 9, 'name': '삼성SDI', 'symbolCode': 'A006400', 'code': 'KR7006400006', 'tradePrice': 567000.0, 'change': 'FALL', 'changePrice': 1000.0, 'changeRate': -0.0017605634, 'marketCap': 38989488510000.0, 'listedShareCount': 68764530, 'foreignRatio': '43.940'}, {'rank': 10, 'name': '카카오', 'symbolCode': 'A035720', 'code': 'KR7035720002', 'tradePrice': 379500.0, 'change': 'RISE', 'changePrice': 12500.0, 'changeRate': 0.0340599455, 'marketCap': 33513750880500.0, 'listedShareCount': 88310279, 'foreignRatio': '31.610'}, {'rank': 11, 'name': 'LG생활건강', 'symbolCode': 'A051900', 'code': 'KR7051900009', 'tradePrice': 1631000.0, 'change': 'RISE', 'changePrice': 20000.0, 'changeRate': 0.0124146493, 'marketCap': 25473279307000.0, 'listedShareCount': 15618197, 'foreignRatio': '45.610'}, {'rank': 12, 'name': '기아차', 'symbolCode': 'A000270', 'code': 'KR7000270009', 'tradePrice': 61100.0, 'change': 'FALL', 'changePrice': 1100.0, 'changeRate': -0.0176848875, 'marketCap': 24767700501700.0, 'listedShareCount': 405363347, 'foreignRatio': '37.510'}, {'rank': 13, 'name': '삼성물산', 'symbolCode': 'A028260', 'code': 'KR7028260008', 'tradePrice': 128000.0, 'change': 'FALL', 'changePrice': 1000.0, 'changeRate': -0.007751938, 'marketCap': 23921546368000.0, 'listedShareCount': 186887081, 'foreignRatio': '13.990'}, {'rank': 14, 'name': 'POSCO', 'symbolCode': 'A005490', 'code': 'KR7005490008', 'tradePrice': 274000.0, 'change': 'RISE', 'changePrice': 5000.0, 'changeRate': 0.0185873606, 'marketCap': 23889192790000.0, 'listedShareCount': 87186835, 'foreignRatio': '50.740'}, {'rank': 15, 'name': '현대모비스', 'symbolCode': 'A012330', 'code': 'KR7012330007', 'tradePrice': 249000.0, 'change': 'FALL', 'changePrice': 1000.0, 'changeRate': -0.004, 'marketCap': 23668618806000.0, 'listedShareCount': 95054694, 'foreignRatio': '43.160'}, {'rank': 16, 'name': 'SK텔레콤', 'symbolCode': 'A017670', 'code': 'KR7017670001', 'tradePrice': 249500.0, 'change': 'RISE', 'changePrice': 3000.0, 'changeRate': 0.0121703854, 'marketCap': 20146054894500.0, 'listedShareCount': 80745711, 'foreignRatio': '33.930'}, {'rank': 17, 'name': '엔씨소프트', 'symbolCode': 'A036570', 'code': 'KR7036570000', 'tradePrice': 864000.0, 'change': 'RISE', 'changePrice': 1000.0, 'changeRate': 0.0011587486, 'marketCap': 18968275008000.0, 'listedShareCount': 21954022, 'foreignRatio': '48.180'}, {'rank': 18, 'name': 'KB금융', 'symbolCode': 'A105560', 'code': 'KR7105560007', 'tradePrice': 45000.0, 'change': 'RISE', 'changePrice': 50.0, 'changeRate': 0.0011123471, 'marketCap': 18711356400000.0, 'listedShareCount': 415807920, 'foreignRatio': '65.460'}, {'rank': 19, 'name': '한국전력', 'symbolCode': 'A015760', 'code': 'KR7015760002', 'tradePrice': 28050.0, 'change': 'FALL', 'changePrice': 250.0, 'changeRate': -0.0088339223, 'marketCap': 18007092359850.0, 'listedShareCount': 641964077, 'foreignRatio': '17.380'}, {'rank': 20, 'name': 'SK이노베이션', 'symbolCode': 'A096770', 'code': 'KR7096770003', 'tradePrice': 188500.0, 'change': 'EVEN', 'changePrice': 0.0, 'changeRate': 0, 'marketCap': 17429758814000.0, 'listedShareCount': 92465564, 'foreignRatio': '24.080'}, {'rank': 21, 'name': '신한지주', 'symbolCode': 'A055550', 'code': 'KR7055550008', 'tradePrice': 33350.0, 'change': 'EVEN', 'changePrice': 0.0, 'changeRate': 0, 'marketCap': 17226169947250.0, 'listedShareCount': 516526835, 'foreignRatio': '57.450'}, {'rank': 22, 'name': 'SK', 'symbolCode': 'A034730', 'code': 'KR7034730002', 'tradePrice': 241000.0, 'change': 'FALL', 'changePrice': 2500.0, 'changeRate': -0.0102669405, 'marketCap': 16956831577000.0, 'listedShareCount': 70360297, 'foreignRatio': '21.420'}, {'rank': 23, 'name': 'LG전자', 'symbolCode': 'A066570', 'code': 'KR7066570003', 'tradePrice': 94700.0, 'change': 'FALL', 'changePrice': 1100.0, 'changeRate': -0.0114822547, 'marketCap': 15497447985800.0, 'listedShareCount': 163647814, 'foreignRatio': '36.330'}, {'rank': 24, 'name': '삼성생명', 'symbolCode': 'A032830', 'code': 'KR7032830002', 'tradePrice': 75700.0, 'change': 'FALL', 'changePrice': 200.0, 'changeRate': -0.0026350461, 'marketCap': 15140000000000.0, 'listedShareCount': 200000000, 'foreignRatio': '12.040'}, {'rank': 25, 'name': 'SK바이오팜', 'symbolCode': 'A326030', 'code': 'KR7326030004', 'tradePrice': 177500.0, 'change': 'FALL', 'changePrice': 4000.0, 'changeRate': -0.0220385675, 'marketCap': 13900601875000.0, 'listedShareCount': 78313250, 'foreignRatio': '5.050'}, {'rank': 26, 'name': '삼성에스디에스', 'symbolCode': 'A018260', 'code': 'KR7018260000', 'tradePrice': 178500.0, 'change': 'RISE', 'changePrice': 1000.0, 'changeRate': 0.0056338028, 'marketCap': 13811937300000.0, 'listedShareCount': 77377800, 'foreignRatio': '11.800'}, {'rank': 27, 'name': 'LG', 'symbolCode': 'A003550', 'code': 'KR7003550001', 'tradePrice': 78000.0, 'change': 'FALL', 'changePrice': 500.0, 'changeRate': -0.0063694268, 'marketCap': 13459456218000.0, 'listedShareCount': 172557131, 'foreignRatio': '34.620'}, {'rank': 28, 'name': '삼성전기', 'symbolCode': 'A009150', 'code': 'KR7009150004', 'tradePrice': 169000.0, 'change': 'RISE', 'changePrice': 1000.0, 'changeRate': 0.005952381, 'marketCap': 12623234624000.0, 'listedShareCount': 74693696, 'foreignRatio': '34.190'}, {'rank': 29, 'name': 'KT&G', 'symbolCode': 'A033780', 'code': 'KR7033780008', 'tradePrice': 88700.0, 'change': 'RISE', 'changePrice': 700.0, 'changeRate': 0.0079545455, 'marketCap': 12177844483900.0, 'listedShareCount': 137292497, 'foreignRatio': '40.590'}, {'rank': 30, 'name': '아모레퍼시픽', 'symbolCode': 'A090430', 'code': 'KR7090430000', 'tradePrice': 208000.0, 'change': 'FALL', 'changePrice': 500.0, 'changeRate': -0.0023980815, 'marketCap': 12159365920000.0, 'listedShareCount': 58458490, 'foreignRatio': '32.150'}], 'totalCount': 1564, 'totalPages': 53, 'currentPage': 1, 'pageSize': 30, 'baseDate': None}

這個測試。

import requests
import time

url = 'https://finance.daum.net/api/trend/market_capitalization?page=1&perPage=30&fieldName=marketCap&order=desc&market=KOSPI&pagination=true'
headers = {
    'referer': 'https://finance.daum.net/domestic/market_cap',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36'}
for i in range(100):
    res = requests.get(url=url, headers=headers)
    for j in res.json()['data']:
        if j['name'] == 'SK하이닉스':
            print(j)
    time.sleep(1)

Output

{'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 116000.0, 'change': 'FALL', 'changePrice': 2500.0, 'changeRate': -0.0210970464, 'marketCap': 84448274340000.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}
{'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 116000.0, 'change': 'FALL', 'changePrice': 2500.0, 'changeRate': -0.0210970464, 'marketCap': 84448274340000.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}
{'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 116000.0, 'change': 'FALL', 'changePrice': 2500.0, 'changeRate': -0.0210970464, 'marketCap': 84448274340000.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}
{'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 115500.0, 'change': 'FALL', 'changePrice': 3000.0, 'changeRate': -0.0253164557, 'marketCap': 84084273157500.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}
{'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 115500.0, 'change': 'FALL', 'changePrice': 3000.0, 'changeRate': -0.0253164557, 'marketCap': 84084273157500.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}
{'rank': 2, 'name': 'SK하이닉스', 'symbolCode': 'A000660', 'code': 'KR7000660001', 'tradePrice': 116000.0, 'change': 'FALL', 'changePrice': 2500.0, 'changeRate': -0.0210970464, 'marketCap': 84448274340000.0, 'listedShareCount': 728002365, 'foreignRatio': '49.810'}
.....

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 如何使用沒有ID標簽或類的Python從網站上刮取表格？ 我無法從該網站上抓取項目。 Python 我正在嘗試從本網站上的 PDF 中抓取標題。 但是，我得到了標題和鏈接。 為什么以及如何解決這個問題？ 我無法弄清楚這是否有問題 我想我在主文件中出現 function 調用錯誤，但我不知道為什么 如何從該站點將團隊名稱和目標刮到表格中？ 我一直在嘗試幾種不同的方法，但無法完全弄清楚 我正在嘗試從網站上抓取表格，但我不斷收到 IndexError 並且無法繼續 無法弄清楚如何使用漂亮的湯（Python）在 body 標簽中抓取數據 我不知道如何阻止它被打印 我如何 web 從包含所有標簽的網站中抓取信息<pre>預格式化的標簽部分？</pre>
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM