How to download files from Scraped Links[python] without logging in to a specified directory

Question

How to Download PDFs from Scraped Links [Python]?

After working on this I was stuck with the following issues Is it possible to download if my college portal has a login page and if yes what should be added to this... Can this be fixed

Answer 1

It is possible, but you should use something like Selenium. This library will allow you to run a browser and navigate through coding, just like you would. You will be able to fill out the username and password forms and click the submit button , and log in to the website. Check out this guide about web scraping and the Selenium library

If you find Selenium too slow, a faster alternative is Mechanize . The problem is that it does not process Javascript, and this will break some webpages.