使用Python在Selenium中使用find_elements_by_class_name查找href属性

Question

Using Selenium with Python.

I've got pages full of links with a class called item-title. I'm trying to iterate through the pages and compile a list of all the link texts and the href attributes that go with them. I want to output the titles and links to a csv file. Here is my code:

myLinks=driver.find_elements_by_class_name("item-title")
for link in myLinks:
    out.write(link.text)
    out.write (",") 
    out.write(link.get_attribute("href"))
    out.write("\n")

The line that outputs the href value gives the following error:

TypeError: expected a character buffer object

Tried the following:

myLinks=driver.find_elements_by_class_name("item-title")
for link in myLinks:
    out.write(link.text)
    out.write (",") 
    out.write(str(link.get_attribute("href")))
    out.write("\n")

Error went away, link text is coming through okay, but now the href is coming through as 'None'

Edit to add the HTML

<div class="item-title">
    <span class="icons-pinned"></span>
    <span class="icons-solved"></span>
    <span class="icons-locked"></span>
    <span class="icons-moved"></span>
    <span class="icons-type"></span>
    <span class="icons-reply"></span>
    <a href="/mylink">My title</a>
</div>

I think I see the issue now. The is a child element of the div, I need to target that, don't I?

Answer 1

As per the HTML you have shared, the link texts and the href attributes are not within the node identified as find_elements_by_class_name("item-title") . Rather they are within the decendent <a> tag. Hence instead of using find_elements_by_class_name("item-title") we have to use either find_elements_by_xpath or find_elements_by_css_selector as follows :

• Using find_elements_by_css_selector :

   myLinks=driver.find_elements_by_css_selector("div.item-title > a") for link in myLinks: out.write(link.get_attribute("innerHTML")) out.write (",") out.write(link.get_attribute("href")) out.write("\\n") 

• Using find_elements_by_xpath :

   myLinks=driver.find_elements_by_xpath("//div[@class='item-title']/a") for link in myLinks: out.write(link.get_attribute("innerHTML")) out.write (",") out.write(link.get_attribute("href")) out.write("\\n")