无法从网站上抓取数据

Question

I want to get the price of a item from this website : https://paytm.com/shop/p/demonio-SUNDEMONIOS-R-193973BC69538C?tracker=%7C%7C%7C%7C%2Fh%2Fbrand-store%2Ffashion-sale-Best-Selling%20Products%7C1

The price is located in the following tag:

<span ng-if="!product.product.isOnlyCarCategory">Buy for Rs 79</span>

I use the following code , but it returns an empty list.

import requests
from bs4 import BeautifulSoup
s=str(raw_input())
r=requests.get(s)
soup=BeautifulSoup(r.content)
item_name=soup.find_all("span",{"ng-if":"!product.product.isOnlyCarCategory"})
print item_name

Answer 1

You can get the json that contains the product data, by appending &callback=angular.callbacks._0&channel=web&version=2 to the URL.

https://catalog.paytm.com/v1/p/demonio-SUNDEMONIOS-R-193973BC69538C?tracker=%7C%7C%7C%7C%2Fh%2Fbrand-store%2Ffashion-sale-Best-Selling%20Products%7C1&callback=angular.callbacks._0&channel=web&version=2

You can then parse the result like this (I assume you are interested in the price):

import json
import requests

r = requests.get(url)
d = json.loads(r.text.split('\n')[1][:-2])
print(d['offer_price'])

The above will give you 79 .