简体   繁体   中英

how to get each element in a class separately in selenium

There is a website that I want to curl, and data is in nested and separated classes and elements.
each page has a different number of these classes and elements. all I want to get data in a list, as each list element is one of the page elements data.

this is what I've done for it :

driver = webdriver.Chrome(chromedriver_path,chrome_options=chrome_options)
driver.get(my_path_to_website)
element = driver.find_element_by_xpath("//div[@class='class1']/*")
driver.execute_script("return arguments[0].textContent;", element)
print(element.text)

but it gives me all the data in the parent class as a string, and I want it separated.

here is the form of HTML in the website:

<div class="class1" id="class1">    
    <div class="b">
        <div class="m1">
            <p>data1</p>
        </div>
        <div class="m2">
            <p>data2</p>
        </div>
    </div>
    <div class="b">
        <div class="m1">
            <p>data3</p>
        </div>
        <div class="m2">
            <p>data4</p>
        </div>
    </div>
.
.
.

</div>

as I said I want my data in a list my_data = ['data 1', 'data 2', 'data 3', 'data 4' , ...] or in a dict ... but with my code, I just get it as a string. my_data = "data 1 data 2 data 3 data 4"

Here is the solution.

my_data = driver.execute_script("var myList=[];arguments[0].forEach(function(element) {myList.push(element.textContent);});return myList;",driver.find_elements_by_css_selector(".class1 p"))
print(my_data)

Here is the output: 在此处输入图片说明

okey, I got it :
in my code instead of find_element_by_xpath i should use find_elements_by_xpath so it returns me a list of elements, and then :

for items in element:
    my_element = items.text
    print (element)

thanks to this post and this post .

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM