使用Beautiful Soup 4和Python解析错误

Question

I need to get the list of the rooms from this website: http://www.studentroom.ch/en/dynasite.cfm?dsmid=106547

I'm using Beautiful Soup 4 in order to parse the page. This is the code I wrote until now:

from bs4 import BeautifulSoup
import urllib

pageFile = urllib.urlopen("http://studentroom.ch/dynasite.cfm?dsmid=106547")
pageHtml = pageFile.read()
pageFile.close()

soup = BeautifulSoup("".join(pageHtml))

roomsNoFilter = soup.find('div', {"id": "ImmoListe"})

rooms = roomsNoFilter.table.find_all('tr', recursive=False)

for room in rooms:
    print room
    print "----------------"

print len(rooms)

For now I'm trying to get only the rows of the table. But I get only 7 rows instead of 78 (or 77).

At first I tough that I was receiving only a partial html, but I printed the whole html and I'm receiving it correctly. There's no ajax calls that loads new rows after the page loaded...

Someone could please help me finding the error?

Answer 1

This is working for me

soup = BeautifulSoup(pageHtml)
div = soup.select('#ImmoListe')[0]
table = div.select('table > tbody')[0]
k = 0
for room in table.find_all('tr'):
    if 'onmouseout' in str(room):
        print room
        k = k + 1
print "Total ",k

Let me know the status