How to extract data from dynamic website?

Question

I am trying to get the restaurant name and address of each restaurant from this platform:

https://customers.dlivery.live/en/list

So far I tried with BeautifulSoup

import requests
from bs4 import BeautifulSoup
import json
url = 'https://customers.dlivery.live/en/list'

headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) '\
           'AppleWebKit/537.36 (KHTML, like Gecko) '\
           'Chrome/75.0.3770.80 Safari/537.36'}

response = requests.get(url,headers=headers)
soup = BeautifulSoup(response.text, "html.parser")
soup

I noticed that within soup there is not the data about the restaurants. How can I do this?

Answer 1

if you inspect element the page, you will notice that the names are wrapped in the card_heading class, and the addresses are wrapped in card_distance class.

soup = BeautifulSoup(response.text, 'html.parser')
restaurantAddress = soup.find_all(class_='card_distance')

for address in restaurantAddress:
   print(address.text)

and

soup = BeautifulSoup(response.text, 'html.parser')
restaurantNames = soup.find_all(class_='card_heading')

for name in restaurantNames:
   print(name.text)

Not sure if this exact code will work, but this is pretty close to what you are looking for.