如何删除重复，包括numpy数组中的字符串？

Question

I want to remove duplicates from numpy array.

>data = np.array([[1,"a",3,3,4],
                  [1,8,9,9,4],
                  [1,"a",3,3,4]])

>new_array = [tuple(row) for row in data]
>uniques = np.unique(new_array)
>uniques

output: array(['1', '3', '4', '8', '9', 'a'], dtype='<U1')

But what I want is

output: np.array([[1,"a",3,3,4],[1,8,9,9,4]])

How can I do this?Thanks.

Answer 1

Use vstack :

print(np.vstack({tuple(row) for row in data}))

Output:

[['1' 'a' '3' '3' '4']
 ['1' '8' '9' '9' '4']]

You can't make them integers because numpy doesn't support mixed-typed data.

Answer 2

NumPy is bad at handling arrays with mixed datatypes, so how about using pandas drop_duplicates instead?

import pandas as pd

data = [[1,"a",3,3,4],[1,8,9,9,4], [1,"a",3,3,4]]
pd.DataFrame(data).drop_duplicates().values

# array([[1, 'a', 3, 3, 4],
#        [1, 8, 9, 9, 4]], dtype=object)