I'm filtering the fixed income products which appears in this website: https://yubb.com.br/investimentos/renda-fixa?investment_type=cdb&months=3&principal=10000000.0&sort_by=minimum_investment
Basically, the page have some cards and I wanna know for each page how many cards appears. For example, choosing cdb as type and 3 months, it shows 16 cards, but with another input of months or type of product, it may appear less cards.
by now, I know how many possible number of pages it will appear looking at "investmentCardContainer__footer", which is a class but the number of cards looks like it shows as style and I dont know how to find it using selenium webdriver.find function.
Here's a tip for what i'm looking for:
The idea it's to get this number of cards and use it in a loop to get the cards information aggregated in a vector.
vetor = ["cdb","lca","lci"]
dataset_boxes =[]
now = time.time()
for i in vetor:
options = Options()
options.add_argument('--headless')
url = 'https://yubb.com.br/investimentos/renda-fixa?investment_type={}&months=12\
&principal=1000000.0&sort_by=net_return'.format(i)
driver = webdriver.Chrome("C:\\Users\\yourpath\\Desktop\\PYTHON\\chromedriver.exe",options=options)
driver.get(url)
time.sleep(1)
num_pages = driver.find_element_by_class_name("investmentCardContainer__footer").text
list_pages = Convert(num_pages)
last_page = int(list_pages[len(list_pages)-3])
driver.quit()
for j in range(1,last_page+1):
url2 = 'https://yubb.com.br/investimentos/renda-fixa?collection_page={}&investment_type={}&months=12\
&principal=1000000.0&sort_by=net_return'.format(j,i)
driver = webdriver.Chrome("C:\\Users\\yourpath\\Desktop\\PYTHON\\chromedriver.exe",options=options)
driver.get(url2)
num_boxes = driver.find_element_by_class_name("investmentCardContainer__body").text
list_boxes = Convert(num_boxes)
dataset_boxes.append(list_boxes)
driver.quit()
print('idk')
later = time.time()
difference = int(later - now)
print('Processo finalizado em {} segundos.'.format(difference))
Use WebDriverWait
and following xpath
to get the no of pages
count.
print(WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,'(//span[@class="page"]//a)[last()]'))).text)
You need to have following imports to execute above code.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
For this link : https://yubb.com.br/investimentos/renda-fixa?investment_type=cdb&months=3&principal=10000000.0&sort_by=minimum_investment
It should return : 8
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.