Scrapy CSS 选择器返回空白

Question

I'm currently trying to scrape the href elements from each restaurant on a website like:

https://www.menulog.com.au/area/2173-moorebank?lat=-33.9477825&long=150.9190988&q=liverpool

The relevant html can be found at:

HTML Snipping

However, when I use the below code in scrapy shell, it returns nothing

response.css("div.c-listing>div>div")

I was wondering why this is the case/ what I can do to resolve this?

Thank you!

Answer 1

use this css selector 'ac-listing-item-link.u-clearfix' to extract url links in scrapy shell.

>>> for url in response.css('a.c-listing-item-link.u-clearfix ::attr("href")').extract():
...     print(response.urljoin(url))
... 
https://www.menulog.com.au/restaurants-blazin-grillz-liverpool/menu
https://www.menulog.com.au/restaurants-phillies-liverpool/menu
https://www.menulog.com.au/restaurants-mcdonalds-liverpool-south/menu
https://www.menulog.com.au/restaurants-kfc-liverpool/menu
https://www.menulog.com.au/restaurants-omer-biryani-house-liverpool/menu
https://www.menulog.com.au/restaurants-classic-burger-liverpool/menu
https://www.menulog.com.au/restaurants-jasmin-1-liverpool/menu
https://www.menulog.com.au/restaurants-subway-liverpool/menu
https://www.menulog.com.au/restaurants-himalayas-indian-restaurant-liverpool/menu
https://www.menulog.com.au/restaurants-jasmins-liverpool/menu
https://www.menulog.com.au/restaurants-sharetea-liverpool/menu
https://www.menulog.com.au/restaurants-healthy-kitchen-liverpool-halal/menu
https://www.menulog.com.au/restaurants-dosa-hut-liverpool/menu
https://www.menulog.com.au/restaurants-the-kulcha-house-liverpool/menu
https://www.menulog.com.au/restaurants-biang-biang-noodle-shop-liverpool/menu
https://www.menulog.com.au/restaurants-zambeekas-liverpool/menu
https://www.menulog.com.au/restaurants-mina-bakery-liverpool/menu
https://www.menulog.com.au/restaurants-crossroads-hotel-liverpool/menu
https://www.menulog.com.au/restaurants-nutrition-station-liverpool/menu
https://www.menulog.com.au/restaurants-mizuki-sushi-liverpool/menu