无法获取所有元素

Question

https://www.whoscored.com/Regions/252/Tournaments/2/Seasons/6335/Stages/13796/Fixtures/England-Premier-League-2016-2017

Trying to capture the ID of every item with "Match Report" (//*[@id="tournament-fixture"]/tbody/tr[2]/td[7]/a[1]) . I can access the visible one's through the source code but the site uses Incapsula, so I am unable to use requests and I'm currently using selenium. I want to access the previous months and when I press the back button it reloads the previous fixtures but the source code doesn't update.

Does anyone know anyway to either get the source code (DataStore.prime) in the previous month using selenium or how to submit a successful requests to gather all fixtures from this particular season?

Answer 1

No need to use selenium, you can access the month data through the url in Chrome Dev tools:

https://www.whoscored.com/tournamentsfeed/13796/Fixtures/?d=201611&isAggregate=false

?d is the year+month, than you get all the data in a list, the first value is id , and you can get Match Report through the id :

https://www.whoscored.com/Matches/1080595/MatchReport
                                  ^^^^id^^