Finding unique elements in nested list

Question

If I have a list mylist = ["[amn,b,c]", "['a,d,e']", "['f,b,e']"] , and I need to a single list with all the unique elements as [amn,b,c,d,e,f] , how can I accomplish that?

I have tried creating a function and also tried some other method, but to no avail.

Function:

mylist = ["[amn,b,c]", "[‘a,d,e’]", "[‘f,b,e’]"]

def print_list(the_list):

for each_item in the_list:

    if isinstance(each_item, list):

        print_list(each_item)

    else:

        print(each_item)

print_list(mylist)

Output:

[amn,b,c]

[‘a,d,e’]

[‘f,b,e’]

Other method:

mylist = ["[amn,b,c]", "[‘a, d,e’]", "[‘f,b,e’]"]

mylist = str(mylist)

mylist = str(mylist)

mylist = [str(x) for x in (mylist)]

mylist = set(mylist)

i = {' ', "'", ',', '[', ']','‘', '’'}

mylist.difference_update(i)

mylist = list(mylist)

mylist.sort()

mylist

Output:

['a', 'b', 'c', 'd', 'e', 'f', 'm', 'n']

Expected Results:

[amn,b,c,d,e,f]

Actual Results:

With the function:

[amn,b,c]

[‘a,d,e’]

[‘f,b,e’]

With the other method:

['a', 'b', 'c', 'd', 'e', 'f', 'm', 'n']

Answer 1

You could use the following list comprehension, where re.sub is used in order to remove unwanted characters, and the underlying lists are obtained using .split , and splitting by , .

Finally in order to obtain the unique elements from the nested list you can use itertools.chain to flatten the nested list, and generate a set from the result in order to keep unique values:

import re
from itertools import chain
set(chain(*[re.sub(r'[\[‘’\"\]]', '', i).split(',') for i in mylist]))
{'a', 'amn', 'b', 'c', 'd', 'e', 'f'}

Where:

[re.sub(r'[\[‘’\"\]]', '', i).split(',') for i in mylist]
[['amn', 'b', 'c'], ['a', 'd', 'e'], ['f', 'b', 'e']]

Answer 2

I had to redefine the list differently because before you had 1 list with strings. If this is wrong let me know, however I am curious as to why you have a list of list-like strings.

mylist = [['amn','b','c'], ['a','d','e'], ['f','b','e']]
unique_list = []

def find_all_unique(input, unique_list):
    if type(input) is list:
        return [find_all_unique(x, unique_list) for x in input if x is not None]
    if type(input) is str:
        if input not in unique_list:
            unique_list.append(input)

find_all_unique(mylist, unique_list)
print(unique_list)

result:

['amn', 'b', 'c', 'a', 'd', 'e', 'f']
[Finished in 0.081s]

or if you need to keep your nested lists within quotations you can use this:

mylist = [['amn','b','c', "['r','t','x']"], ['a','d','e'], ['f','b','e']]
unique_list = []

def find_all_unique(input, unique_list):
    if type(input) is list:
        return [find_all_unique(x, unique_list) for x in input if x is not None]
    if type(input) is str:
        if input.startswith('['):
            temp_list=[]
            exec("temp_list.append(" + input + ')', {"temp_list":temp_list})
            return [find_all_unique(x, unique_list) for x in temp_list if x is not None]
        elif input not in unique_list:
            unique_list.append(input)

find_all_unique(mylist, unique_list)
print(unique_list)

to test this I added a stringed list "['r','t','x']" and this should catch r , t , x as unique inputs

and this results:

['amn', 'b', 'c', 'r', 't', 'x', 'a', 'd', 'e', 'f']
[Finished in 0.077s]

This will work whether it's a list of lists and stringed lists and all since the function is recursive.

Answer 3

Firstly, I would try to substitute the , (comma), ' (single quote), [] (open close square brackets with empty string using pattern matching. Then remove duplicates using set and reconstruct the list using list as below:

my_list = ["[amn,b,c]", "['a, d,e']", "['f,b,e']"]

result = sorted(list(set(([letter for word in my_list for letter in re.sub(',|\'|\[|]|\s+', '', word)]))))

print(result)

where

re.sub(',|\'|\[|]|\s+', '', word)]) 

will replace special characters in the string. For example, ['a, d,e'] to ade .

The comprehension based solution is technically equal to

result = []

for word in my_list:  # Break list of lists to lists
    word = re.sub(',|\'|\[|]|\s+', '', word)
    for letter in word:  # Process each word in the sub list
        result.append(letter)

print('results with duplicates:    ', result)  # List with possible duplicates
result = set(result)  # Remove duplicates by converting to a set

result = list(result)  # Convert set back to list without duplicates (order is not preserved)
print('results without duplicates: ', result)

result = sorted(result)
print('results in sorted order:    ', result)

which results as

results with duplicates:     ['a', 'm', 'n', 'b', 'c', 'a', 'd', 'e', 'f', 'b', 'e']
results without duplicates:  ['e', 'a', 'd', 'm', 'f', 'c', 'n', 'b']
results in sorted order:     ['a', 'b', 'c', 'd', 'e', 'f', 'm', 'n']

Answer 4

dd = []
mylist = ["[amn,b,c]", "[‘a,d,e’]", "[‘f,b,e’]"]
for i in mylist:
    dd.extend([''.join(filter(str.isalnum, j)) for j in i.split(",")])
print (list(set(dd)))
#output ['f', 'a', 'b', 'amn', 'c', 'd', 'e']