In my Python script I am trying to scrape a Twitter page for up to 5 scrolls but my script is throwing a TypeError
stating that WebElements
is not iterable. Any way to solve this issue.
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
path_chromedriver = '/Users/SubrataMohanty/Desktop/chromedriver'
browser = webdriver.Chrome(path_chromedriver)
base_url = u'https://twitter.com/search?q='
query = u'%40sachin_rt'
url = base_url + query
browser.get(url)
time.sleep(1)
body = browser.find_element_by_tag_name('body')
for _ in range(5) :
body.send_keys(Keys.PAGE_DOWN)
time.sleep(0.2)
tweets = browser.find_element_by_class_name('tweet-text')
for tweet in tweets :
print(tweet.text)
Errors :
abhijeet-mohanty-2:Desktop SubrataMohanty$ python tweet_scraper.py
Traceback (most recent call last):
File "tweet_scraper.py", line 23, in <module>
for tweet in tweets :
TypeError: 'WebElement' object is not iterable
find_element_by_class_name()
returns a single element . You meant to find multiple elements by class name - use find_elements_by_class_name()
method (note the s
):
tweets = browser.find_elements_by_class_name('tweet-text')
# HERE^
for tweet in tweets :
print(tweet.text)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.