根据一列值的组合有效拆分 pandas dataframe

Question

Lets say I have a dataframe with one column and it has 3 unique values

Click here to see input

import pandas as pd
df = pd.DataFrame(['a', 'b', 'c'], columns = ['string'])
df

I want to split this dataframe into smaller data frames, such that each dataframe will contain 2 unique values. In the above case I need 3 data frames 3c2(nCr) = 3. df1 - [ab] df2 - [ac] df3 - [bc]. Please click on the below link to see my current implementation.

Click here to see current code and output

import itertools
for i in itertools.combinations(df.string.values, 2):
    print(df[df.string.isin(i)], '\n')

I am looking something like groupby in pandas. Because sub-setting data inside loop is time consuming. In one of the sample case, I have 609 unique values and it was taking around 3 mins to complete the loop. So, looking for some optimized way to perform the same operation, as the unique values may shoot up to 1000's in real scenarios.

Answer 1

It will be slow because you're creating 370k dataframes. If all of them are supposed to only hold two values, why does it need to be a dataframe?

df = pd.DataFrame({'x': range(100)})
df['key'] = 1
records = df.merge(df, on='key').drop('key', axis=1).to_dict('r')
[pd.Series(x) for x in records]

You will see that records is computed quite fast but then it takes a few minutes to generate all of these series objects.