美丽的汤什么也没回报

Question

I am new to Python and I am using beautiful soup to do an assignment about web scraping. The user is asked to input the units of the course. And I should pull out the relevant information about the course including the course title, time, registered students and instructor.

I started with finding the course table which contains all courses information and each course is in a table udder the course table tag. Then I would like to iterate and loop through each course to find out the information. But the code I wrote does not give me anything.

Could anyone take a look at my code? Which part did I do wrong? Thank you in advance. The html link is http://classes.usc.edu/term-20181/classes/itp/ The following is my code to ask for user input and I am trying to use find & find_all function to find the class title,time,students registered and instructor.

from bs4 import BeautifulSoup
import urllib.request


url="http://classes.usc.edu/term-20181/classes/itp/"
page=urllib.request.urlopen(url)

soup=BeautifulSoup(page.read(),"html.parser")

# ask for user input for course units 
choiceUnits=input("Enter")
#trying to find the tag that contain all the courses information
coursesTable=soup.find("div",class_="course-table")  
#trying to find each course table under the course-table tag  
courses=coursesTable.find_all("div",class_="course-info expanded")

for course in courses:
    # trying to find the course units
    unitsTag=courses.find("span",class_="units")
    units=unitsTag.text
    #compare the course units with the user input. If they are the same, find out the course title,time,students registered and instruction 
    if units==choiceUnits:
        #find the title of the course
        titleTag=courses.find("a",class_="courselink")
        title=titleTag.text
        #find the time of the course
        timeTag=courses.find_all("td",class_="time")
        time=timeTag.text
        #find the number of students registered 
        registerTag=courses.find_all("td",class_="registered")
        register=registerTag.text
        #find the instructor 
        instructorTag=courses.find_all("td",class_="instructor")
        instructor=instructorTag.text
        #print out the result to verify 
        print(title)
        print(time,register,instructor)

Answer 1

There are several things in your code that are not working:

The class you use for finding the courses tables course-info expanded does not exists until you click in a course, so you have to use course-info expandable .

Second, you are asking the user for an input of the number of units, but you are extracting the units text which is in a format (#.# units) , so you need to account for that too.

Lastly, in your for loop , you need to access the properties of the course object, and not courses .

This gives the output you want:

from bs4 import BeautifulSoup
import urllib.request


url="http://classes.usc.edu/term-20181/classes/itp/"
page=urllib.request.urlopen(url)
soup=BeautifulSoup(page.read(),"html.parser")

# ask for user input
choiceUnits = float(input("Enter number of units:"))
choiceUnits = "(" + str(choiceUnits) + " units)"

#trying to find the tag that contain all the courses information
coursesTable = soup.find("div",class_="course-table")

#trying to find each course table under the course-table tag
courses = coursesTable.find_all("div",class_="course-info expandable") 
for course in courses:
    # trying to find the course units
    units = course.find("span",class_="units").text

    #compare the course units with the user input.
    #If they are the same, find out the course title,time,students registered and instruction
    if units == choiceUnits:
        title = course.find("a",class_="courselink").text 
        print("Course - {}:".format(title))

        for row in course.find("table", class_="sections responsive").find_all("tr")[1:]:
            time = row.find("td",class_="time").text
            register = row.find("td",class_="registered").text
            instructor = row.find("td",class_="instructor").text

            print(time,register,instructor)