facebook 在 selenium 無頭運行時檢測到瀏覽器自動化

Question

在窗口 chrome 中為 fb 聊天機器人運行此 selenium 腳本時運行良好。 在無頭模式下，即使使用 python 虛擬顯示器，它也會被檢測到並被阻止。

我想使用 pyvirtualdisplay 在無頭模式下運行腳本，同時避免檢測，這樣我就可以作為每日計划任務遠程執行。

這是針對數字海洋上的 ubuntu 18.04 液滴。

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0
from pyvirtualdisplay import Display

display = Display(visible=0, size=(800, 800))
display.start()
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--auto-open-devtools-for-tabs")
chrome_options.add_argument("--load-extension=" + me.favoriteExtension)
chrome_options.add_argument("--start-maximized")
chrome_options.add_experimental_option("prefs", { 
    "profile.default_content_setting_values.notifications": 2
})
chrome_options.add_argument('headless')
chrome_options.add_argument('--no-sandbox')
browser = webdriver.Chrome(chrome_options=chrome_options)

def login(submitBtn):
    email = browser.find_element_by_xpath('//*[@id="email"]')
    password = browser.find_element_by_xpath('//*[@id="pass"]')
    btn = browser.find_element_by_xpath(submitBtn)
    email.send_keys(me.fbUser)
    password.send_keys(me.fbPassword)
    btn.click()

def respondToNewMessages():
    browser.get("http://facebook.com/messages/t/")
    print('responding to new msgs')
    conversations = browser.find_elements_by_xpath('*//ul[@aria-label="Conversation List"]/li')
    print('found '+str(len(conversations)) + ' conversations')
    index = 0
    for conversation in conversations:
        index += 1
        firstMsg = conversation.find_element_by_xpath('./div/a/div/div//span/span')
        print('Found first message ' + firstMsg.text + ' conversation #' + str(index))
        msgFontSize = firstMsg.value_of_css_property('font-weight')
        if(not int(msgFontSize) > 400):
            print('conversation #' + str(index) + ' is an old convo')
        else:
            firstMsg = conversation.find_element_by_xpath('./div/a/div/div//span/span')
            firstMsg.click()
            with open('awaymsg.txt') as f:
                for line in f: 
                    ActionChains(browser).send_keys(line).perform() 
                    ActionChains(browser).send_keys(Keys.ENTER).perform()
            f.close()
    return True

browser.get("http://facebook.com") # load the web page
login("//*[@id='u_0_b' and not(@type='hidden')]|//button[@id='loginbutton' and @type='submit' and @name='login' and not(@type='hidden')]") # Attempt login

while ( 'login' in browser.current_url): # if login fails try again and / or dismiss checkpoint
    login("//*[@id='u_0_b' and not(@type='hidden')]|//button[@id='loginbutton' and @type='submit' and @name='login' and not(@type='hidden')]")
    if 'checkpoint' in browser.current_url:
        browser.find_element_by_xpath("//button[@value='Yes']").click()
    browser.get("http://facebook.com/messages/t/") # exit when messages are loaded

browser.implicitly_wait(5) 
conversations = browser.find_elements_by_xpath('*//ul[@aria-label="Conversation List"]/li')  # count conversations

respondToNewMessages()

browser.close()

預期 - 無頭 w 虛擬顯示將反映在窗口化 chrome 中運行。

實際 - 窗口化運行良好，Facebook 檢測並阻止了無頭。

Answer 1

pyvirtualdisplay和瀏覽器的 headless 模式做同樣的事情，所以如果你使用pyvirtualdisplay那么你不應該設置chrome_options.add_argument('headless') 。

安裝 xvfb ( sudo apt-get install xvfb ) 並將其設置為后端：

display = Display(backend="xvfb", visible=0, size=(800, 800))