Items combination and frequency count in Pandas

Question

I have Dataset like this:

ORDER_CODE	ITEM_ID	ITEM_NAME	TOTALPRICE
123	id1	name1	345
321	id2	name2	678

and Function for calculation which items was sold together. Which ones was most popular or more expensive

out:

ITEM_ID	sold together
id1	[ id33, id23, id12 ]
id2	[ id56, id663 ]

I using this Func:

def freq(df):
    
    hit_list = [list of ID's]
    
    result = pd.DataFrame(columns = ['ITEM_ID', 'sold together'])
    
    unic_arc  = df['ITEM_ID'].unique()
    unic_num = df['ORDER_CODE'].unique()
    data_arc ={}
    data_num={}
    for i in unic_arc:
        data_arc[i] = {}
        
    tturns = response_ur[['ITEM_ID', 'TOTALPRICE']].groupby(by = 'ITEM_ID', as_index = False).sum()
    tturns = tturns.rename(columns = {'ITEM_ID' : 'inum', 'TOTALPRICE' : 'turn'})
    
    for i in tqdm(unic_arc):
        b = df[df['ITEM_ID'] == i]['ORDER_CODE'].values
        for t in b:
            a = df[df['ORDER_CODE'] == t]['ID'].values
            if i in a:
                for arc in a:
                    if int(arc) not in hit_list: 
                        if arc != i:
                            if arc in data_arc[i]:
                                data_arc[i][arc]+=1
                            else:
                                data_arc[i][arc] = 1
                            
        dd = data_arc[i]
                
        tmp = pd.DataFrame(columns = ['inum', 'freq'])
        tmp['inum'] = data_arc[i].keys()
        tmp['freq'] = data_arc[i].values()
        
        tmp['inum'] = tmp['inum'].astype(str)
        tturns['inum'] = tturns['inum'].astype(str)
            
        tmp = pd.merge(tmp, tturns, on = 'inum', how = 'inner')

        tmp = tmp.sort_values(by = ['freq', 'turn'], ascending = False)
        
        if len(tmp['inum'].values) > 14:
            inums = str(tmp['inum'].values[0:15]).replace("\n", "").replace(' ', ',').replace('\'', '')
        else:
            inums = str(tmp['inum'].values).replace("\n", "").replace(' ', ',').replace('\'', '')
            
        result = res.append({'inum' : i, 'recs' : inums}, ignore_index = True)
                            
    return(result)

I try to add merge 1for addint ITEM_NAME in Func on any iteration, but it so long. My dataset have about 10kk rows

I need add to my output one more column with list of 'ITEM_NAME' of 'sold together' list items. And calc it fast?

Answer 1

This might do it:

import pandas as pd

df = pd.DataFrame( {
                    'ORDER_CODE':['123','321','123','123','321','555'], 
                    'ITEM_ID':[1,2,5,5,4,6],
                    'ITEM_NAME':['name1','name2','name3','name4','name5','name6'],
                    'TOTALPRICE':[10,20,50,50,40,60]}
                  )

result = df.groupby("ORDER_CODE").agg({"ITEM_ID":list, "ITEM_NAME":list, "TOTALPRICE":"sum"})

Further good answer how to create a list in a group by aggregation: