So I have following list of links in file called "output":
https://web.archive.org/web/20180101003616/http://onet.pl
https://web.archive.org/web/20180102000139/http://onet.pl
[...]
If you open first link from the list and press "ctrl + f" in firefox, you can find phrase "Katastrofa".
All I want is to have a script, which can find a phrase ("Katastrofa" is only example, I want to use argv argument, but that's not important here), print some success message and proceed further...
I got stuck and can't figure out how to do it. The script I got for testing does not "see" the word ("Katastrofa"), which definitely is on the first page...
Please help :)
Here is what I've done so far:
f = open('output', 'r')
f2 = f.readlines()
for i in f2:
r=requests.get(i)
first_page = r.text
soup = BeautifulSoup(first_page, 'html.parser')
page_soup = soup
fraza = "Katastrofa"
boxes = page_soup.body.find_all(fraza)
print(i)
print(boxes)
Output:
https://web.archive.org/web/20180101003616/http://onet.pl
[]
https://web.archive.org/web/20180102000139/http://onet.pl
[]
https://web.archive.org/web/20180103002217/http://onet.pl
if you want to search if in html string
contain text
for i in f2:
r=requests.get(i)
fraza = "Katastrofa"
if re.match(fraza, r.text, re.I) # ignore case
print(i)
if you want to search html element
contain text
for i in f2:
r=requests.get(i)
soup = BeautifulSoup(r.text, 'html.parser')
fraza = "Katastrofa"
boxes = soup.find_all(True, text=re.compile(fraza, re.I))
if boxes:
print(i)
print(boxes)
Results is list of last child element:
https://web.archive.org/web/20180101003616/http://onet.pl
[<span class="title"> Kostaryka: Katastrofa lotnicza. Media: są ofiary </span>,
<span class="title"> Australia: katastrofa samolotu, są ofiary śmiertelne </span>]
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.