如何从Scrapy响应中的脚本标记中提取Javascript变量？

Question

<script type="text/javascript">

define('roomsAndRatesData', function() {
var roomsAndRatePlans = {"propertyData":{"bookingFeeMessageEnabled":true}},{"rooms":{"24-285501":{"locale":"en"}}}

};
</script>

There where other variable also define in that script. i just need data that are inside "var roomsAndRatePlans" variable can any one suggest solution to get only that variable data.

i have try other solutions but always retrieve all data inside that script tag and converting that data throws error.

so please provide me solution to get only particular data of that script

Thanks

Answer 1

You can use the js2xml library:

>>> import js2xml
>>> from js2xml.utils.vars import get_vars
>>> snippet = '''
var product = {"price": 10.93, "stock": false, "name": "JBL"};
var analytics = {"id": 1234};
'''
>>> get_vars(js2xml.parse(snippet))
{
    'product': {'price': 10.93, 'stock': False, 'name': 'JBL'},
    'analytics': {'id': 1234}
}

Your JS snippet seems to be incomplete, so I used a dummy snippet here. In your case, you may first have to get the JS code via:

>>> snippet = response.css('script:contains("roomsAndRatesData")::text').get()