Python Selenium get list of visible elements too slow

Question

I use python (3.4) and Selenium in order to load a webpage and: first, get all the elements; second, create a list with only elements that are visible. This is my code:

driver = webdriver.Chrome()
driver.maximize_window()
url = "https://www.gazzetta.it/"
driver.get(url)
all_elems = driver.find_elements_by_xpath("//*")
start = datetime.now()
print("Start:  {}".format(start))


visible_elems = []
for elem in all_elems:
    if elem.is_displayed():
        visible_elems.append(elem)

end = datetime.now()
print("End:  {}".format(end))

diff = end - start
print("Diff =  {}".format(diff))    

my problem is that the loop takes forever (on my end, it takes about 1 minute and 20 seconds). I read similar question ( Detect user visible elements(only in viewport) by xpath in selenium, Python , How to create a list of all visible elements in a class python ) but none of them seems to address this specific problem. I know you may wonder why I need all the elements, long story short I upload all the elements inside a dataframe for further analysis. Can somebody think about a way to speed this up? Thank you

Answer 1

Here is a dummy test i did on google.com 10 loops over findElements(by.xpath("//*") and mark if element is displayed.

Found 88 elements
Duration: 00:00:07.001
Found 88 elements
Duration: 00:00:03.952
Found 88 elements
Duration: 00:00:02.740
Found 88 elements
Duration: 00:00:02.579
Found 88 elements
Duration: 00:00:02.566
Found 88 elements
Duration: 00:00:02.532
Found 88 elements
Duration: 00:00:02.694
Found 88 elements
Duration: 00:00:02.554
Found 88 elements
Duration: 00:00:02.419
Found 88 elements
Duration: 00:00:02.436

I don't see any problem with the results.

Note: driver implicit time affects, findElement(s) and the other methods. by default as i remember is 500ms, try manually changing it.