简体   繁体   English

Python和Selenium:访问由Javascript构建的下拉菜单中隐藏的链接有什么好方法?

[英]Python and Selenium: What's a good way to access links hidden in dropdown menus built by Javascript?

I'm building a webcrawler in Python using Selenium. 我正在使用Selenium在Python中构建一个webcrawler。 Below is the function that searches for links. 以下是搜索链接的功能。 It works fine, except for the part that's commented out. 它的工作正常,除了被注释掉的部分。 The idea is to hover over each link that was found on the page being examined, and if that hovering action reveals more links (like in a dropdown menu built with Javascript, for example), then add those links using a recursive call to getLinksFromHTML (the "if code != 1" part is just there to make sure there's only one recursive call). 我们的想法是将鼠标悬停在正在检查的页面上找到的每个链接上,如果该悬停操作显示更多链接(例如在使用Javascript构建的下拉菜单中),则使用递归调用getLinksFromHTML添加这些链接( “if code!= 1”部分就是为了确保只有一个递归调用。

However, the recursive call doesn't pick up any new links when I test it on a page that has links inside JS dropdown menus (the page I'm looking at is http://wilmingtontaxesandaccounting.com ). 但是,当我在JS下拉菜单中有链接的页面上测试时,递归调用不会获取任何新链接(我正在查看的页面是http://wilmingtontaxesandaccounting.com )。 All visible links are picked up fine. 所有可见的链接都很好。

What can I do so that Selenium sees those dropdown links? 我能做些什么让Selenium看到那些下拉链接? I need a general solution, ie, no specific element IDs or anything else page-specific harcoded into the code. 我需要一个通用的解决方案,即没有特定的元素ID或其他任何特定于页面的代码被编码到代码中。 Thanks for reading! 谢谢阅读!

def getLinksFromHTML(currUrl, code):

    ListlinkerHref = browser.find_elements_by_xpath("//*[@href]")
    links1 = []
    links2 = []

    for link in ListlinkerHref:
        url = link.get_attribute("href")

        #hov = ActionChains(browser).move_to_element(link)
        #hov.perform()
        #if code != 1:
            #links1 = self.getLinksFromHTML(currUrl, 1)

        if url not in links1:
            links2.append(url)

    return links1 + links2  

It turns out the reason it wasn't working was that I was using a Firefox driver. 事实证明它不起作用的原因是我使用的是Firefox驱动程序。 Apparently with Firefox, move_to_element doesn't actually hover on the element, it just "focuses" there. 显然使用Firefox,move_to_element实际上并没有悬停在元素上,它只是“聚焦”在那里。 When I switched to a Chrome driver, it actually hovered over the menu items and showed the submenus. 当我切换到Chrome驱动程序时,它实际上悬停在菜单项上并显示子菜单。 (Note the actual function has some errors in it, but that's not the point of this question.) (注意实际函数中有一些错误,但这不是这个问题的重点。)

TL;DR: If you're using Selenium Webdriver and you want to hover over links to reveal content like submenus, use Chrome and not Firefox. TL; DR:如果您正在使用Selenium Webdriver,并且想要将鼠标悬停在链接上以显示子菜单等内容,请使用Chrome而不是Firefox。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM