如何访问此脚本元素的数据？

Question

I'm trying to use requests and BeautifulSoup to access some metadata on a page.

Some script elements can be accessed, but seemingly not one in particular.

For example:

response = BeautifulSoup(requests.get("https://www.booking.com/hotel/br/olympia-residence.en-gb.html", verify=False).content, "html.parser")

scriptData = response.select('script[type="text/javascript"]')

In the HTML, there is a script element with a window.utag_data variable, but scriptData only contains data from another script element.

I thought that the particular script element's absence may be due to it being loaded dynamically on the page, but if that's the case, I couldn't narrow down what response was delivering that data.

Is it possible to get the window.utag_data with requests and BeautifulSoup ?

Answer 1

It seems the website is sending a different HTML depending on how you access the request.

I can see window.utag_data if I access that page from the browser, but not if I fetch it with curl:

$ curl -s https://www.booking.com/hotel/br/olympia-residence.en-gb.html | grep utag_data
$

It also doesn't show in the response downloaded with the code you provided:

>>> 'window.utag_data' in str(response)
False

You can try to replay the request as if it were done by a browser (eg using a browser user agent).