带有 Selenium 和 Node.js 的递归 Facebook 页面 Webscraper

Question

What I try to do is to loop through an array of Facebook page IDs and to return the code from each event page. Unfortunately, I only get the code of the last page ID in the array but as many times as elements are in the array. Eg when I have 3 ID's in the array I get 3 times the code of the last page ID.

I already experimented with async await but I had no success.

The expected outcome would be the code of each page. Thank you for any help and examples.

 //Looping through pages pages.forEach( function(page) { //Creating URL let url = "https://mbasic.facebook.com/"+page+"?v=events"; //Getting URL driver.get(url).then( function() { //Page loaded driver.getPageSource().then(function(result) { console.log(result); }); } ); } );

Answer 1

you faced the same issue i did when i created a scraper using python and selenium. Facebook has countermeasure on manual URL change, you cannot change it , i receive the same data again and again even though it was automated. in order to get a good result you need to have access of face books Graph API which provides a complete object of Facebook page with its pagination URL.

or the second way i got it write was i used on click button of selenium browser automation to scroll down the next page.it wont work like you are typing , i prefer the usage of graph API