如何使用 Beautiful Soup 从网站获取不断变化的数据值？

Question

I am trying web scraping with BeautifulSoup for getting data of BTC-USDT from biance. Actually I am getting what I want but the value is changing in every second in website,but when I am trying to print it to my console it prints me same value and it change rarely. Basically, my data are the same every time when I try to get it, but on the website, it changes every time and I can't get that changing data. What can I do?

from bs4 import BeautifulSoup
import requests
import time



while(True):
    url='https://www.binance.com/tr/trade/BTC_USDT'
    HTML=requests.get(url)
    html_content=HTML.content
    soup=BeautifulSoup(HTML.text,'html.parser')
    paper=str((soup.find('title',attrs={'data-shuvi-head':'true'})))
    print(paper)
    time.sleep(5)

Answer 1

This page uses JavaScript to update data but BeautifulSoup can't run JavaScript . You use need Selenium to control real web browser which can run JavaScript .

from selenium import webdriver
import time
             
url = 'https://www.binance.com/tr/trade/BTC_USDT'  # PEP8: spaces around `=`

#driver = webdriver.Chrome()
driver = webdriver.Firefox()
driver.get(url)

while True:  # PEP8: no need `()`
    try:
        #print(driver.title)
        print(driver.title.split(' ')[0].strip())
    except Exception as ex:
        print('Exception:', ex)
        
    time.sleep(5)

---

Eventually you can check in DevTools (tab Network ) in Chrome / Firefox to see url used by JavaScript to get new data - and then you can try to use it with requests . Because JavaScript usually send data as JSON so you will no need BeautifulSoup but module json .

But first check if you can get it with official Binance API

---

PEP 8 -- Style Guide for Python Code

---

EDIT

Example with Binance API: Current Average Price

import requests
import time

url = 'https://api.binance.com/api/v3/avgPrice?symbol=BTCUSDT'

while True:
    response = requests.get(url)
    data = response.json() 
    print(data['price'])
    
    time.sleep(5)