Selenium: How to select each picture in a dynamically changing page?

Question

My situation is the following: I am writing a bot to automate scrolling through instagrams explore page. I want to like the first 100 pictures that appear for a given hashtag.

I am using: Selenium, python, chrome.

My problem is the following: Whenever I scroll, new posts “appear” and old ones “disappear”. I am using an xpath like this: //article[row]. However, this only works for the first 10 rows since then row 11 is actually row 8 because previous pictures that I have already passed do no longer appear on the same level.

Does somehow know how to handle such a situation? I would like to access every single picture in a dynamically changing page where new pictures are added when I scroll down and older ones disappear.

Thank you so much!

Answer 1

You gotta use a loop.

pseudocode:

posts = []
while something:
   posts.extend(driver.find_posts())
   driver.scroll_till_current_posts_disappeared

for scrolling, at https://stackoverflow.com/a/74508235/20443541

Answer 2

By chance I found an answer to this question: Find next sibling element in Selenium, Python?

Basically you have to use JS: next_sibling = driver.execute_script("""return arguments[0].nextElementSibling """, element)