解析w /区分大小写的文本/标签

Question

I'm parsing through text using beautifulsoup and want to return the tags below/under a parent tag. However, between three different documents there exist inconsistencies between how the 'desired data set' is capitalized. See below:

<td class="pl "...-unimportant bits of script here-...;>Desired Data Set...</td>

and

<td class="pl "...-unimportant bits of script here-...;>Desired data set...</td>

and

<td class="pl "...-unimportant bits of script here-...;>desired data set...</td>

This is my code thus far:

import requests
from bs4 import BeautifulSoup

soup = BeautifulSoup(data.text, 'lxml')

filenames = ['Desired Data Set','desired data set','Desired data set']

for filename in filenames:
    for item in soup.select('filename:contains("' + filename + '")'):
                    for td in item.find('td', text=filename).parent.find_all('td'):
                        data = [td.text.strip()]
                        print(data) 

...and it works.

However, as I start to work with larger data sets, I'm sure there will be even more inconsistencies, and even though the above approach works it's 'hacky' and isn't efficient or prudent. I would like to just use one Filename for all desired data sets.

I've tried to lower the entire soup using lower() but it throws a NoneType error.

Answer 1

You can use string argument of find_all() method:

from bs4 import BeautifulSoup

data = '''<table><tr><td class="pl ">Desired Data Set...</td>
<td class="pl ">Desired data set...</td>
<td class="pl ">desired data set...</td>
<td class="pl ">Something else</td>
</tr></table>
'''

soup = BeautifulSoup(data, 'lxml')

for td in soup.find_all('td', string=lambda t: 'desired data set' in t.lower()):
    print(td)

Prints:

<td class="pl">Desired Data Set...</td>
<td class="pl">Desired data set...</td>
<td class="pl">desired data set...</td>

Answer 2

soup = BeautifulSoup(data.text.lower(), 'lxml')可能是解决问题的“hacky”方式，但对于我的具体示例，它有效。