Try-Catch with Beatifulsoup

Question

So essentially the program I am making is going to grab lyrics out of the genius website for any song the user chooses.

Now the problem I am having is that when I run my program sometimes the lyrics show up, and sometimes they don't and I'm met with an attribute none-type error.

Now I've tried everything I tried using headers which made the none-type error occur more frequently, and even selenium to webscrape but the none-type still occured.

Anyways since I can't get rid of it, I thought of using a try catch error block so that when the non-type error does occur it can just try again.

Now the problem I am having with my try-catch block is that when the none-type error occurs it just goes on an infinite loop, until python gives me the stackoverflow error.

For example I put "loading..." when the none-type error would occur and it just infite loops the "loading...", so how can I fix this error. Any advice would be apperciated.

SongURL = f"https://genius.com{picking()}{token}"
Res_1 = requests.get(SongURL)


soup = BeautifulSoup(Res_1.text, "html5lib")



while True:
    try:
        lyrics = soup.find("div", class_="lyrics").get_text()
        print(lyrics)
        break
    except Exception:
        print("Loading...")
        continue

Answer 1

The problem is that you included a continue within the except part of the block. Catching the exception is the mechanism to tell Python what to do when a problem occurs. When you tell it to continue within a while True , you're telling it to continue with the loop.

I would instead insert another break , and remove the continue . Or, perhaps it's better to re-evaluate whether you need the while loop in the first place. If the goal is simply to retrieve the lyrics once, the loop shouldn't be necessary.