[英]Problem in Selenium with python (instagram crawler)
I just wrote this instagram crawler ,it's a little project for college. 我刚刚写了这个instagram爬虫,这是一个大学的小项目。 I will show you the code and upload a picture to show you what my problem is. 我将向您显示代码并上传图片以显示我的问题是什么。
from time import sleep
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
class App:
def __init__(self,username="Enter your username here",password="Enter your password here",target_username="shriar.ha"):
self.username = username
self.password = password
self.target_username = target_username
self.driver = webdriver.Chrome("/Users/Shahriar/Desktop/Selenium and BS projects/chromedriver.exe") #This is the path to webdriver in my PC ,you should change it and give the path of where your webdriver is located.
self.main_url = "https://www.instagram.com"
self.driver.get(self.main_url)
sleep(5)
self.log_in()
self.close_notification()
self.go_to_target_profile()
sleep(3)
self.click_on_following()
self.move_mouse()
self.scroll_down()
self.driver.close()
def move_mouse(self):
actions = ActionChains(self.driver)
following_list = self.driver.find_element_by_xpath("//div[@class='isgrP']//div[@role = 'button']")
actions.move_to_element(following_list).perform()
sleep(3)
def scroll_down(self):
number_of_following = self.driver.find_element_by_xpath("//a[@href='/shriar.ha/following/']/span").get_attribute("innerHTML")
print(number_of_following)
number_of_following = int(number_of_following)
if number_of_following > 7:
number_of_scrolls = (number_of_following / 7)+3
for i in range(int(number_of_scrolls)):
self.driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
sleep(2)
def click_on_following(self):
following_button = self.driver.find_element_by_xpath("//a[@href='/shriar.ha/following/']")
following_button.click()
sleep(5)
def close_notification(self):
try:
sleep(3)
close_noti_btn = self.driver.find_element_by_xpath("//button[contains(text(),'Not Now')]")
close_noti_btn.click()
sleep(2)
except:
pass
def go_to_target_profile(self):
target_profile_url = self.main_url + "/" + self.target_username + "/"
self.driver.get(target_profile_url)
def log_in(self):
login_button = self.driver.find_element_by_xpath("//a[@href='/accounts/login/?source=auth_switcher']")
login_button.click()
sleep(5)
username_input = self.driver.find_element_by_xpath("//input[@name='username']")
username_input.send_keys(self.username)
password_input = self.driver.find_element_by_xpath("//input[@name='password']")
password_input.send_keys(self.password)
password_input.submit()
if __name__ == "__main__":
app = App()
as you can see, it logins in instagram , then it goes to the target username which you give to the program and then it clicks on following , so it shows the following list . 如您所见,它登录instagram,然后转到您提供给程序的目标用户名,然后单击“ follow”,因此显示以下列表。 this is not completed yet , it should do other things but for now im stuck in this step . 这个还没有完成,它应该做其他的事情,但是现在我停留在这个步骤上。
my problem is that when i click on following . 我的问题是,当我单击以下。 it opens a small window . 它打开一个小窗口。 and that's where you can see the following list , I want to scroll down this list. 那就是您可以在其中看到以下列表的地方,我想向下滚动此列表。 see the picture below : 见下图:
i want to scroll down the following list but instead my code scrolls down the main page , i mean the page that's on the back. 我想向下滚动以下列表,但我的代码却向下滚动主页,我的意思是背面的页面。 I realized that when i get my mouse cursor on the following list , i can scroll it with my mouse, so i decided to write a function to get my mouse cursor on the list and then scroll it , but it was not successful. 我意识到,当我将鼠标光标放在下面的列表上时,可以用鼠标滚动它,因此我决定编写一个函数来将鼠标光标放在列表上,然后滚动它,但是没有成功。
anyone got an idea what ii should do? 任何人都知道我该怎么办?
thanks 谢谢
The following code works fine with me: 以下代码适合我:
def scroll_down(self):
number_of_following = self.driver.find_element_by_xpath("//a[@href='/shriar.ha/following/']/span").get_attribute("innerHTML")
print(number_of_following)
number_of_following = int(number_of_following)
if number_of_following > 7:
number_of_scrolls = (number_of_following / 7)+3
for i in range(int(number_of_scrolls)):
#scroll by element
self.driver.execute_script("arguments[0].scrollIntoView(true)",self.driver.find_element_by_xpath("(//div[@role='dialog']//button[text()='Follow'])["+number_of_following+"]"))
time.sleep(2)
Why are you bothering moving the mouse and clicking? 您为什么要动鼠标并单击呢? You should be able to crawl an account using a library like requests . 您应该能够使用诸如requests之类的库对帐户进行爬网。
Alternatively, there are programs already doing that, that you can get inspired from. 或者,已经有一些程序可以这样做,您可以从中得到启发。
Here are some: 这里有一些:
Unless you are specifically asked for it, I don't think this is a viable solution to move the cursor around and click each link. 除非特别要求您这样做,否则我认为这不是移动光标并单击每个链接的可行解决方案。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.