Scrapy xpath not able to extract

Question

I am trying to extract data fromlink , I used

scrapy shell "https://www.newegg.com/Product/Product.aspx?Item=06T-0045-00045"

I got correct response, but can't get the Xpath to work,ie getting the price with response.xpath('//li[@class="price-current"]') returns empty, I tried also response.xpath('//*[@id="landingpage-price"]/div/div/ul') but also empty, When I use response.xpath('//*[@id="landingpage-price"]') it works but anything deeper returns empty.

Answer 1

You always need to check source HTML ( Ctrl+U ). There is <meta itemprop='price' content='78.23' /> in the source. So simple:

response.xpath('//meta[@itemprop="price"]/@content').extract_first()

will work.

Answer 2

I was searching for the same question for the whole day and find this answer perfect for this

response.xpath('//meta[@itemprop="price"]/@content').get()