简体   繁体   中英

Is there a way to use beautiful soup to filter out the product id embedded in the JavaScript

I am creating a checkout bot for shoes, and want to scrape the data from the JavaScript section in the html source code. I would rather not use string manipulation as it is static. Is there a way to use beautiful soup or something similar to filter out the product information?

This is for a shopify website. I have thought about using findAll("script"), but there are many JavaScript section in the text, and i'd hate to hard code which script section it is.

This is part of the javascript in the raw html. I would like to grab the product id such as "9913856852004" and the shoe size/ name information.

Afterpay.products.push({
  "id": 1063026130980,
  "title": "Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White",
  "handle": "aj6599-202-nike-womens-air-vapormax-fk-moc-2-moon-black white",
  "description": "\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e",
  "published_at": "2018-12-19T13:55:25+11:00",
  "created_at": "2018-12-19T13:55:25+11:00",
  "vendor": "NIKE",
  "type": "Footwear",
  "tags": ["Footwear", "Sneakers"],
  "price": 28000,
  "price_min": 28000,
  "price_max": 28000,
  "available": true,
  "price_varies": false,
  "compare_at_price": null,
  "compare_at_price_min": 0,
  "compare_at_price_max": 0,
  "compare_at_price_varies": false,
  "variants": [{
    "id": 9913856852004,
    "title": "Moon Particle\/Black-White \/ 6",
    "option1": "Moon Particle\/Black-White",
    "option2": "6",
    "option3": null,
    "sku": "36363",
    "requires_shipping": true,
    "taxable": true,
    "featured_image": null,
    "available": true,
    "name": "Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 6",
    "public_title": "Moon Particle\/Black-White \/ 6",
    "options": ["Moon Particle\/Black-White", "6"],
    "price": 28000,
    "weight": 1500,
    "compare_at_price": null,
    "inventory_quantity": 1,
    "inventory_management": "shopify",
    "inventory_policy": "deny",
    "barcode": null
  }, {
    "id": 9913856884772,
    "title": "Moon Particle\/Black-White \/ 7",
    "option1": "Moon Particle\/Black-White",
    "option2": "7",
    "option3": null,
    "sku": "36364",
    "requires_shipping": true,
    "taxable": true,
    "featured_image": null,
    "available": true,
    "name": "Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 7",
    "public_title": "Moon Particle\/Black-White \/ 7",
    "options": ["Moon Particle\/Black-White", "7"],
    "price": 28000,
    "weight": 1500,
    "compare_at_price": null,
    "inventory_quantity": 2,
    "inventory_management": "shopify",
    "inventory_policy": "deny",
    "barcode": null
  }, {
    "id": 9913856983076,
    "title": "Moon Particle\/Black-White \/ 9",
    "option1": "Moon Particle\/Black-White",
    "option2": "9",
    "option3": null,
    "sku": "36367",
    "requires_shipping": true,
    "taxable": true,
    "featured_image": null,
    "available": false,
    "name": "Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 9",
    "public_title": "Moon Particle\/Black-White \/ 9",
    "options": ["Moon Particle\/Black-White", "9"],
    "price": 28000,
    "weight": 1500,
    "compare_at_price": null,
    "inventory_quantity": 0,
    "inventory_management": "shopify",
    "inventory_policy": "deny",
    "barcode": null
  }, {
    "id": 9913857015844,
    "title": "Moon Particle\/Black-White \/ 10",
    "option1": "Moon Particle\/Black-White",
    "option2": "10",
    "option3": null,
    "sku": "36368",
    "requires_shipping": true,
    "taxable": true,
    "featured_image": null,
    "available": true,
    "name": "Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 10",
    "public_title": "Moon Particle\/Black-White \/ 10",
    "options": ["Moon Particle\/Black-White", "10"],
    "price": 28000,
    "weight": 1500,
    "compare_at_price": null,
    "inventory_quantity": 1,
    "inventory_management": "shopify",
    "inventory_policy": "deny",
    "barcode": null
  }],
  "images": ["\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200", "\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_2.jpg?v=1545188202", "\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_3.jpg?v=1545188203", "\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_4.jpg?v=1545188206"],
  "featured_image": "\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200",
  "options": ["Color", "Size"],
  "content": "\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e"
});

I would like something that could grab the id from the javascript information dynamically.

You could use a combination of BeautifulSoup , json and re . It is really hard to answer without the actual url, but here is something that might work. You can tailor this to suit your specific use case.

Case 1: each product is in different script tag.

html="""
<script>
Afterpay.products.push({"id":1063026130980,"title":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White","handle":"aj6599-202-nike-womens-air-vapormax-fk-moc-2-moon-black white","description":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e","published_at":"2018-12-19T13:55:25+11:00","created_at":"2018-12-19T13:55:25+11:00","vendor":"NIKE","type":"Footwear","tags":["Footwear","Sneakers"],"price":28000,"price_min":28000,"price_max":28000,"available":true,"price_varies":false,"compare_at_price":null,"compare_at_price_min":0,"compare_at_price_max":0,"compare_at_price_varies":false,"variants":[{"id":9913856852004,"title":"Moon Particle\/Black-White \/ 6","option1":"Moon Particle\/Black-White","option2":"6","option3":null,"sku":"36363","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 6","public_title":"Moon Particle\/Black-White \/ 6","options":["Moon Particle\/Black-White","6"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856884772,"title":"Moon Particle\/Black-White \/ 7","option1":"Moon Particle\/Black-White","option2":"7","option3":null,"sku":"36364","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 7","public_title":"Moon Particle\/Black-White \/ 7","options":["Moon Particle\/Black-White","7"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":2,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856983076,"title":"Moon Particle\/Black-White \/ 9","option1":"Moon Particle\/Black-White","option2":"9","option3":null,"sku":"36367","requires_shipping":true,"taxable":true,"featured_image":null,"available":false,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 9","public_title":"Moon Particle\/Black-White \/ 9","options":["Moon Particle\/Black-White","9"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":0,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913857015844,"title":"Moon Particle\/Black-White \/ 10","option1":"Moon Particle\/Black-White","option2":"10","option3":null,"sku":"36368","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 10","public_title":"Moon Particle\/Black-White \/ 10","options":["Moon Particle\/Black-White","10"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null}],"images":["\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_2.jpg?v=1545188202","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_3.jpg?v=1545188203","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_4.jpg?v=1545188206"],"featured_image":"\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","options":["Color","Size"],"content":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e"});
</script>
<script>
something before this
Afterpay.products.push({"id":1063026130981,"title":"Nike Men's Air Vapormax FK Moc 2 - Moon\/Black\/White","handle":"aj6599-202-nike-womens-air-vapormax-fk-moc-2-moon-black white","description":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e","published_at":"2018-12-19T13:55:25+11:00","created_at":"2018-12-19T13:55:25+11:00","vendor":"NIKE","type":"Footwear","tags":["Footwear","Sneakers"],"price":28000,"price_min":28000,"price_max":28000,"available":true,"price_varies":false,"compare_at_price":null,"compare_at_price_min":0,"compare_at_price_max":0,"compare_at_price_varies":false,"variants":[{"id":9913856852004,"title":"Moon Particle\/Black-White \/ 6","option1":"Moon Particle\/Black-White","option2":"6","option3":null,"sku":"36363","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 6","public_title":"Moon Particle\/Black-White \/ 6","options":["Moon Particle\/Black-White","6"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856884772,"title":"Moon Particle\/Black-White \/ 7","option1":"Moon Particle\/Black-White","option2":"7","option3":null,"sku":"36364","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 7","public_title":"Moon Particle\/Black-White \/ 7","options":["Moon Particle\/Black-White","7"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":2,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856983076,"title":"Moon Particle\/Black-White \/ 9","option1":"Moon Particle\/Black-White","option2":"9","option3":null,"sku":"36367","requires_shipping":true,"taxable":true,"featured_image":null,"available":false,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 9","public_title":"Moon Particle\/Black-White \/ 9","options":["Moon Particle\/Black-White","9"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":0,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913857015844,"title":"Moon Particle\/Black-White \/ 10","option1":"Moon Particle\/Black-White","option2":"10","option3":null,"sku":"36368","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 10","public_title":"Moon Particle\/Black-White \/ 10","options":["Moon Particle\/Black-White","10"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null}],"images":["\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_2.jpg?v=1545188202","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_3.jpg?v=1545188203","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_4.jpg?v=1545188206"],"featured_image":"\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","options":["Color","Size"],"content":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e"});
</script>
<script>
We don't need this
</script>
"""
from bs4 import BeautifulSoup
import re
import json
soup=BeautifulSoup(html,"html.parser")
scripts=soup.find_all("script",text=re.compile('Afterpay.products.push'))
product_info_list=[]
pattern = re.compile(r"\((.*?)\)")
for script in scripts:
    json_string=pattern.search(script.text).group(1)
    product_info_list.append(json.loads(json_string))
#print the items
for product in product_info_list:
    print(product['id'])
    print(product['title'])
    print("--------------------")

Outputs:

1063026130980
Nike Women's Air Vapormax FK Moc 2 - Moon/Black/White
--------------------
1063026130981
Nike Men's Air Vapormax FK Moc 2 - Moon/Black/White
--------------------

Case 2: If they are all within a single script tag.

html="""
<script>
Afterpay.products.push({"id":1063026130980,"title":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White","handle":"aj6599-202-nike-womens-air-vapormax-fk-moc-2-moon-black white","description":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e","published_at":"2018-12-19T13:55:25+11:00","created_at":"2018-12-19T13:55:25+11:00","vendor":"NIKE","type":"Footwear","tags":["Footwear","Sneakers"],"price":28000,"price_min":28000,"price_max":28000,"available":true,"price_varies":false,"compare_at_price":null,"compare_at_price_min":0,"compare_at_price_max":0,"compare_at_price_varies":false,"variants":[{"id":9913856852004,"title":"Moon Particle\/Black-White \/ 6","option1":"Moon Particle\/Black-White","option2":"6","option3":null,"sku":"36363","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 6","public_title":"Moon Particle\/Black-White \/ 6","options":["Moon Particle\/Black-White","6"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856884772,"title":"Moon Particle\/Black-White \/ 7","option1":"Moon Particle\/Black-White","option2":"7","option3":null,"sku":"36364","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 7","public_title":"Moon Particle\/Black-White \/ 7","options":["Moon Particle\/Black-White","7"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":2,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856983076,"title":"Moon Particle\/Black-White \/ 9","option1":"Moon Particle\/Black-White","option2":"9","option3":null,"sku":"36367","requires_shipping":true,"taxable":true,"featured_image":null,"available":false,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 9","public_title":"Moon Particle\/Black-White \/ 9","options":["Moon Particle\/Black-White","9"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":0,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913857015844,"title":"Moon Particle\/Black-White \/ 10","option1":"Moon Particle\/Black-White","option2":"10","option3":null,"sku":"36368","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 10","public_title":"Moon Particle\/Black-White \/ 10","options":["Moon Particle\/Black-White","10"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null}],"images":["\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_2.jpg?v=1545188202","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_3.jpg?v=1545188203","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_4.jpg?v=1545188206"],"featured_image":"\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","options":["Color","Size"],"content":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e"});
Afterpay.products.push({"id":1063026130981,"title":"Nike Men's Air Vapormax FK Moc 2 - Moon\/Black\/White","handle":"aj6599-202-nike-womens-air-vapormax-fk-moc-2-moon-black white","description":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e","published_at":"2018-12-19T13:55:25+11:00","created_at":"2018-12-19T13:55:25+11:00","vendor":"NIKE","type":"Footwear","tags":["Footwear","Sneakers"],"price":28000,"price_min":28000,"price_max":28000,"available":true,"price_varies":false,"compare_at_price":null,"compare_at_price_min":0,"compare_at_price_max":0,"compare_at_price_varies":false,"variants":[{"id":9913856852004,"title":"Moon Particle\/Black-White \/ 6","option1":"Moon Particle\/Black-White","option2":"6","option3":null,"sku":"36363","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 6","public_title":"Moon Particle\/Black-White \/ 6","options":["Moon Particle\/Black-White","6"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856884772,"title":"Moon Particle\/Black-White \/ 7","option1":"Moon Particle\/Black-White","option2":"7","option3":null,"sku":"36364","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 7","public_title":"Moon Particle\/Black-White \/ 7","options":["Moon Particle\/Black-White","7"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":2,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913856983076,"title":"Moon Particle\/Black-White \/ 9","option1":"Moon Particle\/Black-White","option2":"9","option3":null,"sku":"36367","requires_shipping":true,"taxable":true,"featured_image":null,"available":false,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 9","public_title":"Moon Particle\/Black-White \/ 9","options":["Moon Particle\/Black-White","9"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":0,"inventory_management":"shopify","inventory_policy":"deny","barcode":null},{"id":9913857015844,"title":"Moon Particle\/Black-White \/ 10","option1":"Moon Particle\/Black-White","option2":"10","option3":null,"sku":"36368","requires_shipping":true,"taxable":true,"featured_image":null,"available":true,"name":"Nike Women's Air Vapormax FK Moc 2 - Moon\/Black\/White - Moon Particle\/Black-White \/ 10","public_title":"Moon Particle\/Black-White \/ 10","options":["Moon Particle\/Black-White","10"],"price":28000,"weight":1500,"compare_at_price":null,"inventory_quantity":1,"inventory_management":"shopify","inventory_policy":"deny","barcode":null}],"images":["\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_2.jpg?v=1545188202","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_3.jpg?v=1545188203","\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_4.jpg?v=1545188206"],"featured_image":"\/\/cdn.shopify.com\/s\/files\/1\/0104\/1862\/products\/Wmns_Vapormax_Moc_Moon_1.jpg?v=1545188200","options":["Color","Size"],"content":"\u003cp\u003eNike continue to modify the coveted Vapormax silhouette, bringing us the Vapormax Moc 2. The shoe features a lightweight FlyKnit upper with a gore strap in place of the regular lacing system, sitting atop the signature Vapormax midsole. Other details include synthetic overlays for added durability, heel pull tabs and subtle swoosh branding.\u003cbr\u003e\u003cbr\u003eSizes are shown in women's US. For men's sizing minus 1.5.\u003c\/p\u003e"});
</script>
<script>
We don't need this
</script>
"""
from bs4 import BeautifulSoup
import re
import json
soup=BeautifulSoup(html,"html.parser")
product_info_list=[]
product_strings=re.findall(r'\(.*\)',soup.text)
for product_string in product_strings:
    product_info_list.append(json.loads(product_string[1:-1]))

for product in product_info_list:
    print(product['id'])
    print(product['title'])
    print('------------------------')

Outputs:

1063026130980
Nike Women's Air Vapormax FK Moc 2 - Moon/Black/White
------------------------
1063026130981
Nike Men's Air Vapormax FK Moc 2 - Moon/Black/White
------------------------

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM