Python Beautiful Soup停止解析

Question

I try to parse the attached text.txt file (with html syntax) by the following script.

#!/usr/bin/python3

import re
from bs4 import BeautifulSoup

pattern = re.compile("www.geocaching.com")
f=open("text.txt")
text=f.read()
f.close()
s = BeautifulSoup(text)
a = s.find_all(href=pattern)
print(len(a))
print (a[len(a)-1])

My expectation is to have all tags with href="www.geocaching.com", but I do not get all from the file attached. The last one is:

<a class="lnk " href="http://www.geocaching.com/geocache/GC3HWHJ_corse-known-unknown-2-view-on-ile-de-giraglia"><span>Corse known &amp; unknown 2 - View on Ile de Giraglia</span></a>

if I delete the lines 626-674, containing only some simple html code, I get the next two, ie the last is

<a class="lnk " href="http://www.geocaching.com/geocache/GC3MEDG_tour-genoise-dagnello"><span>TOUR GENOISE D'AGNELLO</span></a>

but again I don't get all results I can find manually in the html file.

The file I use is from here (I downloaded it to use it locally) https://www.geocaching.com/seek/nearest.aspx?lat=43.410333&lon=09.0476&dist=100

Answer 1

Try to use CSS Selector instead in the following way :

from bs4 import BeautifulSoup

f = open("text.txt")
text = f.read()
f.close()

soup = BeautifulSoup(text)

# this find all the href containing the text "www.geocaching.com"
links =  soup.select('[href]~="www.geocaching.com"')