如何使用 BeautifulSoup 和 Requests 抓取动态变量 Javascript 值

Question

I am scraping login page, i only need VAR SALT= variable in JAVASCRIPT TAG. This is the website = https://ib.muamalatbank.com/ib-app/loginpage

When i am read all answer here,using BeautifulSoup and requests, i can get these 2 variable(Maybe because its static): var muserid='User ID must be filled'; var mpassword= 'Password must be filled';

But when i try Scrape this var SALT= , its give me all VAR value. My result code in python

I just need This VAR SALT value only with no Quotation mark Here the PIC = Source VAR SALT VALUE

I already using re.search, and re.compile, re.findall, but i am Newbie, keep gives me error "Object cannot string...."

from bs4 import BeautifulSoup as bs
import requests
import re
import lxml
import json

URL = 'https://ib.muamalatbank.com/ib-app/loginpage'
REF = 'https://ib.muamalatbank.com'

HEADERS = {'User-Agent': 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:81.0) Gecko/20100101 Firefox/81.0', 'origin': URL, 'referer': REF}

s = requests.session()
soup = bs(s.get(URL, headers=HEADERS, timeout=5, verify=False).text,"html.parser")

script = soup.find_all("script")[11]
ambilteks = soup.find_all(text=re.compile("salt=(.*?)"))
print(ambilteks)

Note: 1) i need Help but not interested using Selenium,

1. I have script in PHP-Laravel, its fully working(i need in Python), but i have no knowledge in laravel, anyone can ask me , i will give the Laravel code

Please help me, thank you very much

Answer 1

Try using re.compile and add the '' into your regex, then extract first result. Not tested with page response. First verify the string is actually present in the response.

p = re.compile(r"var salt='(.*?)'")
res = p.findall(s.get(URL, headers=HEADERS, timeout=5, verify=False).text)[0]
print(res)