Python Re.Search：如何在两个字符串之间找到 substring，该字符串还必须包含特定的 substring

Question

I am writing a little script to get my F@H user data from a basic HTML page.

I want to locate my username on that page and the numbers before and after it.

All the data I want is between two HTML <tr> and </tr> tags.

I am currently using this:

re.search(r'<tr>(.*?)</tr>', htmlstring)

I know this works for any substring, as all google results for my question show. The difference here is i need it only when that substring also contains a specific word

However that only returns the first string between those two delimiters, not even all of them.

This pattern occurs hundreds of times on the page. I suspect it doesn't get them all because I'm not handling all the newline characters correctly but I'm not sure.

If it would return all of them, I could at least then sort them out to find one that contains my username going through each result.group() , but I can't even do that.

I have been fiddling with different regex expressions for ages now but can't figure what one I need to much frustration.

TL;DR - I need a re.search() pattern that finds a substring between two words, that also contains a specific word.

Answer 1

If I understand correctly something like this might work
<tr>(?:(?:(?:(?.<\/tr>)?)*?)\bWORD\b(:.?*?))<\/tr>

• <tr> find "<tr>"
• (?:(?:(?.<\/tr>)?)*?) Find anything except "</tr>" as few times as possible
• \bWORD\b find WORD
• (?:.*?)) find anything as few times as possible
• <\/tr> find "</tr>"

Sample

Answer 2

There are a few ways to do it but I prefer the pandas way:

from urllib import request

import pandas as pd # you need to install pandas

base_url = 'https://apps.foldingathome.org/teamstats/team3446.html'

web_request = request.urlopen(url=base_url).read()

web_df: pd.DataFrame = pd.read_html(web_request, attrs={'class': 'members'})
web_df = web_df[0].set_index(keys=['Name'])
# print(web_df)

user_name_to_find_in_table = 'SteveMoody'
user_name_df = web_df.loc[user_name_to_find_in_table]
print(user_name_df)

Then there are plenty of ways to do this. Using just Beautifulsoup find or css selectors, or maybe re as Peter suggest?

Using beautifulsoup and "find" method, and re, you can do it the following way:

import re
from bs4 import BeautifulSoup as bs # you need to install beautifullsoup
from urllib import request




base_url = 'https://apps.foldingathome.org/teamstats/team3446.html'

web_request = request.urlopen(url=base_url).read()

page_soup = bs(web_request, 'lxml') # need to install lxml and bs4(beautifulsoup for Python 3+)

user_name_to_find_in_table = 'SteveMoody'

row_tag = page_soup.find(
    lambda t: t.name == "td"
              and re.findall(user_name_to_find_in_table, t.text, flags=re.I)
).find_parent(name="tr")

print(row_tag.get_text().strip('tr'))

Using Beautifulsoup and CSS Selectors(no re but Beautifulsoup):

from bs4 import BeautifulSoup as bs # you need to install beautifulsoup
from urllib import request


base_url = 'https://apps.foldingathome.org/teamstats/team3446.html'

web_request = request.urlopen(url=base_url).read()

page_soup = bs(web_request, 'lxml') # need to install lxml and bs4(beautifulsoup for Python 3+)

user_name_to_find_in_table = 'SteveMoody'

row_tag = page_soup.select_one(f'tr:has(> td:contains({user_name_to_find_in_table})) ')

print(row_tag.get_text().strip('tr'))

In your case I would favor the pandas example as you keep headers and can easily get other stats, and it runs very quickly.

Using Re:

So fa, best input is Peters' comment Link , so I just adapted it to Python code (happy to get edited), as this solution doesn't need any extra libraries installation.

import re
from urllib import request




base_url = 'https://apps.foldingathome.org/teamstats/team3446.html'

web_request = request.urlopen(url=base_url).read()
user_name_to_find_in_table = 'SteveMoody'
re_patern = rf'<tr>(?:(?:(?:(?!<\/tr>).)*?)\{user_name_to_find_in_table}\b(?:.*?))<\/tr>'
res = re.search(pattern=re_patern, string= str(web_request))

print(res.group(0))

Helpful lin to use variables in regex: stackflow