从字符串列表中删除相似的重复项

Question

I'm trying to remove the similar duplicates from my list. Here is my code:

l = ["shirt", "shirt", "shirt len", "pant", "pant cotton", "len pant", "watch"]

res = [*set(l)]
print(res)

This will Remove only shirt word which is actually duplicate, but I'm looking to remove the similar words to remove like shirt Len,pant cotton,Len pant. Like that.

Expecting Output as Shirt,pant,watch

Answer 1

It sounds like you want to check if the single-word strings are in any other string, and if so remove them as a duplicate. I would go about it this way:

• Separate the list into single-word strings and any other string.
• For each longer string, check if any of the single-word strings is contained in it.
  • If so, remove it. Otherwise, add it to the result.
• Finally, add all the single-word strings to the result.

l = ["shirt", "shirt", "shirt len", "pant", "pant cotton", "len pant", "watch"]

single, longer = set(), set()
for s in l:
    if len(s.split()) == 1:
        single.add(s)
    else:
        longer.add(s)

res = set()
for s in longer:
    if not any(word in s for word in single):
        res.add(s)
res |= single

print(res)

This example will give:

{'shirt', 'watch', 'pant'}

Answer 2

You can try something like below:

by selecting single word element from list and then apply set

lst = ["shirt", "shirt", "shirt len", "pant cotton", "len pant", "watch"]
set([ls for ls in lst if ' 'not in ls]) 
#Output {'pant', 'shirt', 'watch'}

note if your input will ["shirt", "shirt", "shirt len", "pant cotton", "len pant", "watch"] then output will be {'shirt', 'watch'}

and if still would like to add pant, cotton then you can try

set(sum([ls.split(' ') for ls in lst], []))
#output {'cotton', 'len', 'pant', 'shirt', 'watch'}

and later filter out word by conditions as per your requirements