There is a website that I want to curl, and data is in nested and separated classes and elements.
each page has a different number of these classes and elements. all I want to get data in a list, as each list element is one of the page elements data.
this is what I've done for it :
driver = webdriver.Chrome(chromedriver_path,chrome_options=chrome_options)
driver.get(my_path_to_website)
element = driver.find_element_by_xpath("//div[@class='class1']/*")
driver.execute_script("return arguments[0].textContent;", element)
print(element.text)
but it gives me all the data in the parent class as a string, and I want it separated.
here is the form of HTML in the website:
<div class="class1" id="class1">
<div class="b">
<div class="m1">
<p>data1</p>
</div>
<div class="m2">
<p>data2</p>
</div>
</div>
<div class="b">
<div class="m1">
<p>data3</p>
</div>
<div class="m2">
<p>data4</p>
</div>
</div>
.
.
.
</div>
as I said I want my data in a list my_data = ['data 1', 'data 2', 'data 3', 'data 4' , ...]
or in a dict ... but with my code, I just get it as a string. my_data = "data 1 data 2 data 3 data 4"
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.