在 Python 上用 Selenium/BS 抓取 Linkedin SalesNavigator

Question

我正在嘗試使用 Selenium 和 python 上的 BS 抓取 Sales Navigator 鏈接的結果。我首先登錄 Linkedin，然后使用 driver.get(link) 訪問有問題的 SalesNav 搜索 URL。 不幸的是，該鏈接無緣無故將我重定向到https://www.linkedin.com/sales/contract-chooser並無限期地加載，如果我只是將鏈接復制粘貼到我的瀏覽器上，效果很好。 這是我的代碼：

# Sign in to LinkedIn account
driver = webdriver.Chrome("/Applications/chromedriver")
driver.get("https://linkedin.com/uas/login")
time.sleep(5)
  
username = driver.find_element_by_id("username")
username.send_keys("EMAIL")  # Enter Your Email Address
  
pword = driver.find_element_by_id("password")
pword.send_keys("PASSWORD")


driver.find_element_by_xpath("//button[@type='submit']").click()


driver.maximize_window()
time.sleep(10)

# Redirect to Page
link="https://www.linkedin.com/sales/search/people#logHistory=true&query=(recentSearchParam%3A(id%3A1560694714%2CdoLogHistory%3Atrue)%2Cfilters%3AList((type%3AREGION%2Cvalues%3AList((id%3A101282230%2Ctext%3AGermany%2CselectionType%3AINCLUDED)))%2C%2C%2C(type%3ACURRENT_COMPANY%2Cvalues%3AList((id%3A109842%2Ctext%3Aneckermann.de%20GmbH%2CselectionType%3AINCLUDED)%2C(id%3A12756300%2Ctext%3AFRIEDRICH%20VORWERK%20Unternehmensgruppe%2CselectionType%3AINCLUDED)%2C(id%3A9837432%2Ctext%3ABuffalo%20Boots%20GmbH%2CselectionType%3AINCLUDED)%2C(id%3A12481732%2Ctext%3AYazaki%20Systems%2CselectionType%3AINCLUDED)%2C(id%3A5860104%2Ctext%3ATurck%20Beierfeld%20GmbH%2CselectionType%3AINCLUDED)%2C(id%3A36436702%2Ctext%3ALUX%20Automation%20GmbH%2CselectionType%3AINCLUDED)%2C(id%3A41395024%2Ctext%3AGEDIA%20Gebr%C3%BCder%20DINGERKUS%20GmbH%2CselectionType%3AINCLUDED)%2C(id%3A41053846%2Ctext%3APAUL%20WURTH%20DEUTSCHLAND%20GMBH%2CselectionType%3AINCLUDED)%2C(id%3A3239211%2Ctext%3AArcandor%20AG%2CselectionType%3AINCLUDED)%2C(id%3A36565653%2Ctext%3ASport%20Voswinkel%20GmbH%20%26%20Co.%20KG%2CselectionType%3AINCLUDED)%2C(id%3A37277217%2Ctext%3Aaccredis%20Inkasso%20GmbH%20%26%20Co.%20KG%2CselectionType%3AINCLUDED)%2C(id%3A69449704%2Ctext%3ASch%C3%A4fer%20Informationstechnik%20GmbH%2CselectionType%3AINCLUDED)%2C(text%3AASSET%20Immobilienbeteiligungen%20GmbH%2CselectionType%3AINCLUDED)%2C(text%3AGothaer%20Systems%20GmbH%2CselectionType%3AINCLUDED)%2C(text%3AIFORM%20Spritzgu%C3%9Fformen%20f%C3%BCr%20die%20Kunststoffverarbeitung%20GmbH%2CselectionType%3AINCLUDED)))%2C(type%3ASENIORITY_LEVEL%2Cvalues%3AList((id%3A1%2Ctext%3AUnpaid%2CselectionType%3AEXCLUDED)%2C(id%3A2%2Ctext%3ATraining%2CselectionType%3AEXCLUDED)))%2C(type%3ATITLE%2Cvalues%3AList((text%3AAnwendung%2CselectionType%3AINCLUDED)%2C(text%3Aapplication%2CselectionType%3AINCLUDED)%2C(text%3Aarbeitsplatz%2CselectionType%3AINCLUDED)%2C(text%3Aautomation%2CselectionType%3AINCLUDED)%2C(text%3AAutomatisierung%2CselectionType%3AINCLUDED)%2C(text%3ABeachtung%2CselectionType%3AINCLUDED)%2C(text%3Acdo%2CselectionType%3AINCLUDED)%2C(text%3Acio%2CselectionType%3AINCLUDED)%2C(text%3Aciso%2CselectionType%3AINCLUDED)%2C(text%3Acloud%2CselectionType%3AINCLUDED)%2C(text%3Acollaboration%2CselectionType%3AINCLUDED)%2C(text%3Acompliance%2CselectionType%3AINCLUDED)%2C(text%3Acontainer%2CselectionType%3AINCLUDED)%2C(text%3Acontainer-orchestration%2CselectionType%3AINCLUDED)%2C(text%3Acontainer-orchestrierung%2CselectionType%3AINCLUDED)%2C(text%3Acontainerisieren%2CselectionType%3AINCLUDED)%2C(text%3Acontainerize%2CselectionType%3AINCLUDED)%2C(text%3Acto%2CselectionType%3AINCLUDED)%2C(text%3Adata%2CselectionType%3AINCLUDED)%2C(text%3Adaten%2CselectionType%3AINCLUDED)%2C(text%3Adatenschutz%2CselectionType%3AINCLUDED)%2C(text%3Adigital%2CselectionType%3AINCLUDED)%2C(text%3Adigital%2520transformation%2CselectionType%3AINCLUDED)%2C(text%3Adigitale%2520Transformation%2CselectionType%3AINCLUDED)%2C(text%3ADocker%2CselectionType%3AINCLUDED)%2C(text%3Afertigung%2CselectionType%3AINCLUDED)%2C(text%3AHerstellung%2CselectionType%3AINCLUDED)%2C(text%3Ahr%2CselectionType%3AINCLUDED)%2C(text%3Ahuman%2520resources%2CselectionType%3AINCLUDED)%2C(text%3AHumanressourcen%2CselectionType%3AINCLUDED)%2C(text%3AIndustrie%25204.0%2CselectionType%3AINCLUDED)%2C(text%3Aindustry%25204.0%2CselectionType%3AINCLUDED)%2C(text%3Ainformation%2CselectionType%3AINCLUDED)%2C(text%3AInformationen%2CselectionType%3AINCLUDED)%2C(text%3Ainnovation%2CselectionType%3AINCLUDED)%2C(text%3AInternet%2520der%2520Dinge%2CselectionType%3AINCLUDED)%2C(text%3Ainternet%2520of%2520things%2CselectionType%3AINCLUDED)%2C(text%3Aiot%2CselectionType%3AINCLUDED)%2C(text%3Ait%2CselectionType%3AINCLUDED)%2C(text%3AKubernetes%2CselectionType%3AINCLUDED)%2C(text%3ALeistung%2CselectionType%3AINCLUDED)%2C(text%3Amanufacturing%2CselectionType%3AINCLUDED)%2C(text%3AMenschen%2CselectionType%3AINCLUDED)%2C(text%3Anetwork%2CselectionType%3AINCLUDED)%2C(text%3Anetzwerk%2CselectionType%3AINCLUDED)%2C(text%3Apeople%2CselectionType%3AINCLUDED)%2C(text%3Aperformance%2CselectionType%3AINCLUDED)%2C(text%3APersonalwesen%2CselectionType%3AINCLUDED)%2C(text%3Aproduction%2CselectionType%3AINCLUDED)%2C(text%3Aproduktion%2CselectionType%3AINCLUDED)%2C(text%3Aprotection%2CselectionType%3AINCLUDED)%2C(text%3Aschutz%2CselectionType%3AINCLUDED)%2C(text%3Asecurity%2CselectionType%3AINCLUDED)%2C(text%3ASicherheit%2CselectionType%3AINCLUDED)%2C(text%3Asoftware%2CselectionType%3AINCLUDED)%2C(text%3Atechnologie%2CselectionType%3AINCLUDED)%2C(text%3Atechnology%2CselectionType%3AINCLUDED)%2C(text%3Atransformation%2CselectionType%3AINCLUDED)%2C(text%3AWolke%2CselectionType%3AINCLUDED)%2C(text%3Aworkplace%2CselectionType%3AINCLUDED)%2C(text%3Aprogram%2CselectionType%3AINCLUDED)%2C(text%3Aprokurist%2CselectionType%3AINCLUDED)%2C(text%3Asolution%2CselectionType%3AINCLUDED)%2C(text%3Acyber%2CselectionType%3AINCLUDED)%2C(text%3Aentwicklung%2CselectionType%3AINCLUDED)%2C(text%3Asupply%2520chain%2CselectionType%3AINCLUDED)%2C(text%3Ainfrastructure%2CselectionType%3AINCLUDED)%2C(text%3AInfrastruktur%2CselectionType%3AINCLUDED)%2C(text%3AVersorgungskette%2CselectionType%3AINCLUDED))%2CselectedSubFilter%3ACURRENT)%2C(type%3AFUNCTION%2Cvalues%3AList((id%3A13%2Ctext%3AInformation%2520Technology%2CselectionType%3AINCLUDED)%2C(id%3A12%2Ctext%3AHuman%2520Resources%2CselectionType%3AINCLUDED)%2C(id%3A21%2Ctext%3APurchasing%2CselectionType%3AINCLUDED)%2C(id%3A8%2Ctext%3AEngineering%2CselectionType%3AINCLUDED)%2C(id%3A19%2Ctext%3AProduct%2520Management%2CselectionType%3AINCLUDED)))))&rsLogId=1056057746&searchSessionId=HslSmbZJS06pJuDD8v5%2F0g%3D%3D&sessionId=7tfxUO0tQjSSh52UN4Uvww%3D%3D&viewAllFilters=true"


driver.get(link)

Answer 1

Selenium 用於點擊元素，在文本框中輸入內容等。如果你想抓取內容，你不應該使用 selenium，而是 beautifulsoup。