用beautifulsoup │python 3.8 从天才歌词中获取歌曲歌词

Question

I am trying to get the lyrics for a song from genius lyrics using beautifulsoup but when trying to print out the lyrics, I get no output. Here is my code:

import requests 
from bs4 import BeautifulSoup
songURL = requests.get("https://genius.com/Marshmello-and-bastille-happier-lyrics")
song = songURL.content
soup = BeautifulSoup(song, 'lxml')
lyrics = soup.find_all("section")
for lyr in lyrics:
    for lyr1 in lyrics.select("p"):
        print(lyr1.text)      

Why is this not working, can somebody please look into this as I have been trying to do this for a while now.

Answer 1

It seems that the server returns two version of the page: in one version there are tags with class="song_body-lyrics" , in other version with class="Lyrics__Container..." .

This script tries to handle both cases:

import requests 
from bs4 import BeautifulSoup

url = 'https://genius.com/Marshmello-and-bastille-happier-lyrics'
soup = BeautifulSoup(requests.get(url).content, 'lxml')

for tag in soup.select('div[class^="Lyrics__Container"], .song_body-lyrics p'):
    t = tag.get_text(strip=True, separator='\n')
    if t:
        print(t)

Prints:

[Intro]
Lately, I've been, I've been thinking
I want you to be happier, I want you to be happier
[Verse 1]

...and so on.

Answer 2

import requests 
from bs4 import BeautifulSoup
songURL = requests.get("https://genius.com/Marshmello-and-bastille-happier-lyrics")
song = songURL.content
soup = BeautifulSoup(song, 'lxml')
final_lyrics = []
lyrics = soup.find('div', {'class': "lyrics"})
lyrics = lyrics.find_all('p')
for i in lyrics:
    final_lyrics.append(i.text)
    print(i)

Answer 3

you should get all texts those are in a specific div. you can find that specific div with devtools or viewsource in your browser. here that specific div is <div class='lyrics'> the unique feature of this div is its class, that is class 'lyrics' so we should find this specific div in HTML and then print all texts in that div.

import bs4 as bs
import urllib.request

source = urllib.request.urlopen('https://alirezaarabi.com/view-source_https___genius.com_Alessia-cara-ready-lyrics.html').read()

soup = bs.BeautifulSoup(source,'lxml')
print(soup.title.string)

for div in soup.find_all('div', class_='lyrics'):
    print(div.text)

Answer 4

If you take a look at the actual HTML source code, there are no section tags. Here's what the structure actually looks like:

<div class="song_body column_layout" initial-content-for="song_body">
  <div class="column_layout-column_span column_layout-column_span--primary">
    <div class="song_body-lyrics">
      
        <h2 class="text_label text_label--gray text_label--x_small_text_size u-top_margin">Happier Lyrics</h2>
      
      <div initial-content-for="lyrics">
        <div class="lyrics">
          
            <!--sse-->
            <p>[Intro]<br>
Lately, I've been, I've been thinking<br>
I want you to be happier, I want you to be happier<br>
<br>
...