Better way to find existence of arrays in list of arrays

Question

I am a newbie to Python and trying out different ways to optimize and simplify my code.

I have a list of arrays(necessarily in this format) initially empty, which I need to update with arrays, making sure that duplicate entries are not added.

Right now I am doing it the following way, which is the only thing i tried out which works:

if len(where(((array(self.pop_next)-(self.pop[self.top_indv_indx[i]]))==0).sum(1)==len((self.pop[self.top_indv_indx[i]])))[0])<=0):
     self.pop_next.append(self.pop[self.top_indv_indx[i]])

where self.pop_next is my list of arrays and self.pop[self.top_indv_indx[i]] is the array to be added.

I know this Unpythonic and guess that there are much better simple ways to do the same. Please Help

Answer 1

Edit: I see from your comment that you're using numpy arrays. I've never used numpy so I have no idea how they work with sets.

One option would be to use a set . Sets are like lists but they are unordered and only allow each item to be added once:

>>> s = set()
>>> s.add(1)
>>> s.add(2)
>>> s.add(2)
>>> s.add(2)
>>> s
set([1, 2])

However, you'll run into problems if you try to add a list to a set:

>>> s.add(['my','list'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

An item must be hashable to add to set , and a list can't be hashable as it can't have an unchanging hash value since it can be modified at any time by adding or removing values.

If you don't need the lists you are checking to be mutable you can convert them to tuples which are fixed and so hashable and so set-friendly:

>>> mylist = ['my','list']
>>> s = set()
>>> s.add(tuple(mylist))
>>> s.add(tuple(mylist))
>>> s
set([('my', 'list')])

Answer 2

You may want to try with numpy.all(array1 == array2) as condition for an individual array comparison.

Extension in edit:

To loop over the list, you may use the following:

if all((numpy.all(array_to_add != a) for a in array_list)):
    array_list.append(array_to_add)

This compares array_to_add to all elements of array_list by value. Note that all here is __builtin__.all , in contrast to numpy.all . If you did from numpy import * before, this will not work. Use import numpy instead and call functions by full name as in the example above.

If it is ok to compare by object (ie two arrays are only the same if the are the exact same object in memory), use the following simpler variant:

if array_to_add is not in array_list:
    array_list.append(array_to_add)