清理数据：如何遍历列表查找项目是否包含字符串，空格或空格，然后在Python中删除该项目

Question

I am trying to iterate through a list of data to clean it up.

Here's a small part of the list:

lines =['Wirkstoffliste 1 –  ','','  ', 'Gaschromatographie (GC) ', 'LOQ ', '[mg/kg] ', 'Acibenzolar-S-methyl', 'Aclonifen', 'Acrinathrin', 'Alachlor', 'Aldrin', 'Allethrin', 'Ametryn', 'Antrachinon', 'Atrazin', 'Atrazin-desethyl', 'Atrazin-desisopropyl', 'Azinphos (-ethyl)', 'Azinphos-methyl', 'Benalaxyl', 'Benfluralin', 'Benzoylprop-ethyl',' Seite 13 von 14 ', '   ', ' ', ' ', 'Wirkstoffliste 4 - ','Version 7.2 ']

I want to remove any list item that contains the words "Version", "Seite" and "Wirkstoffliste". You will also see there are some strings that are either blank or contain just white space (of various lengths).

I have already cleaned this data up quite a lot with regex, but now I just want the chemical names. There are some other items that keep coming up that I don't want, eg "Version" but they are never quite the same, so it might be "Version 7. 2" or "Version 8.1". Therefore I thought if I tried "If 'Version' in string", this would find it within the string, then I could choose to delete it. However this doesn't seem to work.

Do I really need to use regex with this too?

Here's a bunch of stuff I tried.

I have tried if string in item.

if "Wirkstoffliste" in item:
    lines.remove(item)

I have tried using OR logic so I could put more search strings in there. eg

if "Seite" or "Wirkstoffliste" or "Version" in item:
    lines.remove(item)

I used both enumerate with del and and if in statement, eg

for n,item in enumerate(lines):
    if "Wirkstoffliste" in item:
        del lines[n]

And finally I tried using a list of search strings:

removables=["Seite","Version","Wirkstoffliste","Gaschromatographie","LOQ"]

for line in lines:
    for r in removables:
        if r in line:
            lines.remove(line)

To delete the blanks and white spaces I have tried:

"""delete empty items"""
lines = list(filter(None, lines))
lines = list(filter(bool,lines))

and

for item in lines:
    if item=="" or " ":
        lines.remove(item)

I have found none of the above works, so I am a little confused what I am doing wrong.

Answer 1

here is a solution: i am using filter and any

l1 = ['Wirkstoffliste', 'Seite','Version']
#i am with lines[:] (slicing) to play with the fact a list is mutable
lines[:] = list(filter(str.strip,lines)) #suppress items whitespace or empty
lines[:] = [x for x in lines if not any(sub in x for sub in l1)]

# you could write these lines too if using a new list:
#lines = list(filter(str.strip,lines))
#lines = [x for x in lines if not any(sub in x for sub in l1)]
print(lines)

output:

['Gaschromatographie (GC) ', 'LOQ ', '[mg/kg] ', 'Acibenzolar-S-methyl', 
 'Aclonifen', 'Acrinathrin', 'Alachlor', 'Aldrin', 'Allethrin', 'Ametryn', 
 'Antrachinon', 'Atrazin', 'Atrazin-desethyl', 'Atrazin-desisopropyl', 
 'Azinphos (-ethyl)', 'Azinphos-methyl', 'Benalaxyl', 
 'Benfluralin', 'Benzoylprop-ethyl']

Another way to write the coding using filter: filter keeps the data if return function is True

def remove_whitespaces_and_items(item):
    if item.strip() == '': return False # if item is blank, dont keep
    for x in l1:
        if x in item:
            return False                # if item of l1 is in lines, dont keep
    return True                         # item is not blank and not in l1, so keep it

lines =list(filter(remove_whitespaces_and_items,lines))

Answer 2

I'm just aa simple man, and going with what you tried, I did a code I think more human readable:

words = ['Wirkstoffliste', 'Seite', 'Version', '  ']
new_lines = []
for item in lines:
    if not (any(word in item for word in words)):
        if item != "" and item != " ":
            new_lines.append(item)

You can add anything in words. (I just inserted 2 blank spaces to catch 2-3-4 spaces fields). I Think for the lines you provided and purpose you wanted, "mg/kg" would be one.

By the way Frenchy version is surely better and more elegant.