如何从 BS4 刮取字符串

Question

recently I scraped a text/javascript that contains the following code:

var spConfigDisabledProducts = [-1
        , '294653', '294655', '294656', '294657', '294658', '294659', '294660', '294661', '294662', '294663', '294664', '294666', '294667', '294668', '294669', '294670', '294671', '294672', '294673'        ];
        {"attributes":{"959":{"id":"959","code":"aw_taglia","label":"Taglia","options":[{"id":"1717","label":"15","price":"0","oldPrice":"0"...

I just want to get all the numbers inside var spConfigDisabledProducts excluded -1, so I tried this:

js = soup.find_all('script')[25].text.replace(',}', '}').replace(',]', ']').strip()

js = json.dumps(js)
obj = json.loads(js)

data_oos = obj.split('var spConfigDisabledProducts = [-1,')
data_oos = data_oos[1].split("];")

But it returns the entire javascript, not only var spConfigDisabledProducts.

How can I fix this? Thanks in advance

Answer 1

You could regex out string representation of list and convert to actual list then slice

import re, json, ast

s = '''var spConfigDisabledProducts = [-1
        , '294653', '294655', '294656', '294657', '294658', '294659', '294660', '294661', '294662', '294663', '294664', '294666', '294667', '294668', '294669', '294670', '294671', '294672', '294673'        ];
        {"attributes":{"959":{"id":"959","code":"aw_taglia","label":"Taglia","options":[{"id":"1717","label":"15","price":"0","oldPrice":"0"'''

p = re.compile(r'spConfigDisabledProducts = (\[[\s\S]*?\])')
data = ast.literal_eval(p.findall(re.sub('\n|\s{2,}','',s))[0])
print(data[1:])