简体   繁体   中英

Iterating over website table rows and fetch data

How can I iterate and scrape every existing row in this website: https://icostats.com/ ?

Is it possible to do it with something like the code below?

rows = []
for row in rows(0, 20):
    row += 1
    get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div:nth-child({})").format(row)

Full code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait

def get_css_sel(selector):
    posts = browser.find_elements_by_css_selector(selector)
    for post in posts:
        print(post.text)

browser = webdriver.Chrome(executable_path=r'C:\Users\alph1\Scrapers\chromedriver.exe')
browser.get("https://icostats.com")
wait(browser, 40).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div:nth-child(2) > div:nth-child(8)")))

browser.execute_script('''
    var element = document.getElementsByClassName("buyNow-0-81"), index;
    for (index = element.length - 1; index >= 0; index--) {
    element[index].parentNode.removeChild(element[index]);
    }
''')

get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tableheader-0-50")              #fetch header of table

rows = []
for row in rows(0, 20):
    row += 1
    get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div:nth-child({})").format(row)

忘记循环,只需执行以下操作:

get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div")

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM