Python BeautifulSoup 循环遍历 div 和多个元素

Question

I have a website containing film listings, I've put together a simplified HTML of the website. Please note that for the real world example the <ul> tags are not direct children of the class film_listing or showtime. They are found under several <div> or <ul> elements.

<li class="film_listing">
       <h3 class="film_title">James Bond</h3>
       <ul class="showtimes">
              <li class="showtime">
                     <p class="start_time">15:00</p>
              </li>
              <li class="showtime">
                     <p class="start_time">19:00</p>
                     <ul class="attributes">
                            <li class="audio_desc">
                            </li>
                            <li class="open_cap">
                            </li>
                     </ul>
              </li>
       </ul>
</li>

I have created a Python script to scrape the website which currently lists all film titles with the first showtime and first attribute of each. However, I am trying to list all showtimes. The final aim is to only list film titles with open captions and the showtime of those open captions performances.

Here is the python script with a nested for loop that doesn't work and prints all showtimes for all films, rather than showtimes for a specific film. It is also not set up to only list captioned films yet. I suspect the logic may be wrong and would appreciate any advice. Thanks!

for i in soup.findAll('li', {'class':'film_listing'}):
    film_title=i.find('h3', {'class':'film_title'}).text  
    print(film_title)
 
    for j in soup.findAll('li', {'class':'showtime'}):
            print(j['showtime.text'])   

    #For the time listings, find ones with Open Captioned
    i=filmlisting.find('li', {'class':'open_cap'})
    print(film_access)

edit: small correction to html script

Answer 1

There are many ways how you could extract the information. One way is to "search backwards" . Search for <li> with class="open_cap" and the find previous start time and film title:

from bs4 import BeautifulSoup


txt = '''
<li class="film_listing">
       <h3 class="film_title">James Bond</h3>
       <ul class="showtimes">
              <li class="showtime">
                     <p class="start_time">15:00</p>
              </li>
              <li class="showtime">
                     <p class="start_time">19:00</p>
                     <ul class="attributes">
                            <li class="audio_desc">
                            </li>
                            <li class="open_cap">
                            </li>
                     </ul>
              </li>
       </ul>
</li>'''

soup = BeautifulSoup(txt, 'html.parser')


for open_cap in soup.select('.open_cap'):
    print('Name       :', open_cap.find_previous(class_='film_title').text)
    print('Start time :', open_cap.find_previous(class_='start_time').text)
    print('-' * 80)

Prints:

Name       : James Bond
Start time : 19:00
--------------------------------------------------------------------------------

Answer 2

Content of read.html

<li class="film_listing">
  <h3 class="film_title">James Bond</h3>
  <ul class="showtimes">
    <li class="showtime">
      <p class="start_time">15: 00</p>
    </li>
    <li class="showtime">
      <p class="start_time">19:00</p>
      <ul class="attributes">
        <li class="audio_desc"></li>
        <li class="open_cap"></li>
      </ul>
    </li>
  </ul>
</li>

As you said <ul> tags are not direct children of the class film_listing or showtime then you can try find() to get first element with specified tag name or you can use find_all() to get list of elements with specified tag name. You can try this

    from bs4 import BeautifulSoup as bs
    
    text = open("read.html", "r")
    
    soup = bs(text.read(), 'html.parser')
    
    for listing in soup.find_all("li", class_="film_listing"):
        print("Film name: ", listing.find("h3", class_="film_title").text)
        print("Start time: ", listing.find("p", class_="start_time").text)
   

Output:

Film name:  James Bond
Start time:  15: 00

instead of find() you can use find_all() method which will return all the tags which that name <p> and class start_time