如何在pyhton中有效地创建一组集合？

Question

I have two dataframes two dataframes, with two columns. The rows are value pairs, where order is not important: ab == ba for me. I need to compare these value pairs between the two dataframes. I have a solution, but that is terribly slow for a dataframe with 300k

import pandas as pd

df1 = pd.DataFrame({"col1" : [1,2,3,4], "col2":[2,1,5,6]})
df2 = pd.DataFrame({"col1" : [2,1,3,4], "col2":[1,9,8,9]})

mysets = [{x[0],x[1]} for x in df1.values.tolist()]
df1sets = []
for element in mysets:
    if element not in df1sets:
        df1sets.append(element)
           
mysets = [{x[0],x[1]} for x in df2.values.tolist()]
df2sets = []
for element in mysets:
    if element not in df2sets:
        df2sets.append(element)

intersect_sets = [x for x in df1sets if x in df2sets]

this works, but it is terribly slow, and there must be an easier way to do this. One of my problem is that is that I cannot add a set to a set, I cannot create {{1,2}, {2,3}} etc

Answer 1

Pandas solution is merge with sorted values of columns, remove duplicates and convert to sets:

intersect_sets = ([set(x) for x in pd.DataFrame(np.sort(df1.to_numpy(), axis=1))
                        .merge(pd.DataFrame(np.sort(df2.to_numpy(), axis=1)))
                        .drop_duplicates()
                        .to_numpy()])
       
print (intersect_sets)
[{1, 2}]

Another idea with set of frozensets:

intersect_sets = (set([frozenset(x) for x in df1.to_numpy()]) & 
                  set([frozenset(x) for x in df2.to_numpy()]))
print (intersect_sets)
{frozenset({1, 2})}