简体   繁体   English

我无法抓取该 div 中的信息

[英]I can't scrape the information inside that div

the list Li should have the 116 elements but instead it has 0 elements列表 Li 应该有 116 个元素,但它有 0 个元素

import requests
from bs4 import BeautifulSoup
url='https://www.profile.nl/zoekresultaten/banden?size_width=205&size_height=60&size_inch=16'
profile= requests.get(url)
if profile.ok:

    soup = BeautifulSoup(profile.text,'lxml')

    Li= (soup.body).findAll("div", attrs={"class": "row xs-ptb-1"})
    print(len(Li))

It is being dynamically pulled from another endpoint you can see in network tab.它是从另一个端点动态拉出的,您可以在网络选项卡中看到。 With a browser this additional request is used to update the page.对于浏览器,此附加请求用于更新页面。 With requests, this does not happen;对于请求,这不会发生; so you make the request direct to that other endpoint to get the info.因此您将请求直接发送到其他端点以获取信息。

import requests

r = requests.get('https://www.profile.nl/tyres_search.json?locale=nl&size_width=205&size_height=60&size_inch=16.0').json()
print(r['products'])

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM