刮谷歌地图审查一家公司的文本数据

Question

I wanted to scrape text review data from google maps review for one company in order to perform sentiment analysis. However, my code is not running. I am getting error. I was wondering if you could guide me to fix this. Thanks!

!pip install selenium
!apt-get update 
!apt install chromium-chromedriver

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
driver =webdriver.Chrome('chromedriver',chrome_options=chrome_options)

#add your google map link whose data you want to scrape
from selenium import webdriver                       
from bs4 import BeautifulSoup                       
import time                       
import io                       
import pandas as pd

from selenium import webdriver
from bs4 import BeautifulSoup
import time
import io
import pandas as pd
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC  
from selenium.webdriver.common.by import By  

driver.get('https://www.google.com/maps/place/Embassy+of+Bangladesh/@38.9418017,-77.0679642,15z/data=!4m7!3m6!1s0x0:0x5621455e7625f36e!8m2!3d38.9418017!4d-77.0679642!9m1!1b1')

wait = WebDriverWait(driver, 10)
menu_bt = wait.until(EC.element_to_be_clickable(
                       (By.XPATH, '//button[@data-value=\'Sort\']'))
                   )  
menu_bt.click()
recent_rating_bt = driver.find_elements_by_xpath(
                                     '//div[@role=\'menuitem\']')[50]
recent_rating_bt.click()
time.sleep(5)

Error message:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-35-94b4c6e89470> in <module>()
      5 menu_bt.click()
      6 recent_rating_bt = driver.find_elements_by_xpath(
----> 7                                      '//div[@role=\'menuitem\']')[50]
      8 recent_rating_bt.click()
      9 time.sleep(5)

IndexError: list index out of range

Answer 1

You're accessing the item indexed by 50 on the list returned by find_elements_by_xpath(). The error message indicates that this index does not exist, ie the returned list is smaller than that.

You should check the length of the returned list before accessing it.