PYTHON：在 html 源代码中查找隐藏元素

Question

I'm trying get the url below which is hidden inside the page source code at https://www.aliexpress.com/item/32212764152.html but it's hidden inside a script tag.

<script>
                    window.runParams = {"descriptionModule":{"descriptionUrl":"https://aeproductsourcesite.alicdn.com/product/description/pc/v2/en_US/desc.htm?productId=32212764152&key=HTB1GwO_aVY7gK0jSZKzM7OikpXac.zip&token=f32528ddd34e37aecddda1c7778d5f0c"} .... </script>

I've manage to get the source code but not sure how to extract the url as an object.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
import re

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
CHROMEDRIVER_PATH = '/Users/reezalaq/PycharmProjects/wholesale/driver/chromedriver'
options = Options()
options.headless = False
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)
driver.get('https://www.aliexpress.com/item/32212764152.html')

html = driver.page_source

def run_script():
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    body = driver.find_element_by_css_selector('body')
    body.send_keys(Keys.PAGE_UP)

count = 0
while count < 3: #13
     run_script()
     count+=1
     time.sleep(5)

x = html.startswith('https://aeproductsourcesite.alicdn.com')
print(x)

How do I filter everything else in the source code and have an object?

x = " https://aeproductsourcesite.alicdn.com/product/description/pc/v2/en_US/desc.htm?productId=32212764152&key=HTB1GwO_aVY7gK0jSZKzM7OikpXac.zip&token=f32528ddd34e37aecddda1c7778d5f0c "

Answer 1

You can use Regular Expression to extract the value:

import re
#..
url = re.compile(r'"descriptionUrl":"([^"]*)"').search(html).group(1)