Extract unique list from nested list in Python

Question

I want to extract unique data from nested list, see below. I implemented two way of this. First one works good, but second one failed. Is new_data is empty during calculation? And how do I fix it?

 data = [                                                                                                                                      
     ['a', 'b'],                                                                                                                               
     ['a', 'c'],                                                                                                                               
     ['a', 'b'],                                                                                                                               
     ['b', 'a']                                                                                                                                
 ]                                                                                                                                             

 # working                                                                                                                                          
 new_data = []                                                                                                                                 
 for d in data:                                                                                                                                
     if d not in new_data:                                                                                                                     
         new_data.append(d)                                                                                                                    
 print(new_data)                                                                                                                               
 # [['a', 'b'], ['a','c'], ['b','a']]                                                                                                          

 # Failed to extract unique list                                                                                                                                 
 new_data = []                                                                                                                                 
 new_data = [d for d in data if d not in new_data]                                                                                             
 print(new_data)                                                                                                                               
 # [['a', 'b'], ['a', 'c'], ['a', 'b'], ['b', 'a']] 

Answer 1

Just try:

new_data = [list(y) for y in set([tuple(x) for x in data])]

You cannot use set() on a list of lists because lists are not hashable. You convert the list of lists into a list of tuples. Apply set() to remove the duplicates. Then convert the de duplicated list of tuples back into a list of lists.

Answer 2

您可以使用enumerate来测试当前值之前是否没有副本，以便仅采用副本的第一个实例：

new_data = [item for index, item in enumerate(data) if item not in data[:index]]