Scraping Yelp for Motel Reviews

Question

EDIT: I have code now using the Yelp API, but it only gives me three reviews per motel. How can I adjust my code to provide more reviews? I'm more interested in having a larger sample size of reviews than of motels.

Code below:

import requests
import json

api_key = 'my api key is here'
headers = {'Authorization': 'Bearer %s' % api_key}

url = 'https://api.yelp.com/v3/businesses/search'
params = {'term':'motel','location':'my city is here'}

req = requests.get(url, params=params, headers=headers)

parsed = json.loads(req.text)

businesses = parsed["businesses"]

for business in businesses:
     print("Name:", business["name"])
     print("Rating:", business["rating"])
     print("Address:", " ".join(business["location"]["display_address"]))
     print("Phone:", business["phone"])
     print("\n")

id = business["id"]

url="https://api.yelp.com/v3/businesses/" + id + "/reviews"

req = requests.get(url, headers=headers)

parsed = json.loads(req.text)

reviews = parsed["reviews"]

print("--- Reviews ---")

for review in reviews:
    print("User:", review["user"]["name"], "Rating:", review["rating"], "Review:", review["text"], "\n")

I am very new to coding and am having trouble finding code that I can make work for my task at hand. I need to scrape Yelp reviews (or TripAdvisor if that is easier) from multiple different motels. I would be looking at a geographic area (my current city) and scraping the reviews from the motels in that area. Then, I need to search the reviews for certain key terms. Any recommendations?

Answer 1

I would suggest using the requests library in Python and sending API requests using Yelp's API!

https://www.yelp.com/developers/documentation/v3

If memory serves, you'll need to sign up to get an API key from Yelp. Whole process is rather quick.