简体   繁体   中英

Python/Pandas: combine columns from 2 dataframes based on match of values between columns, but can't use merge

I have two dataframes and I need to separate rows where a value from pmdf matches one of the codes in jcrdf.All_codes.

If pmdf.code is in jcrdf.All_codes, I need a dataframe with all values from jcrdf AND pmdf.count.

Dataframes:

pmdf = pd.DataFrame(
        {
        'code': ['0567-8315','0007-4977','0096-0225','1365-2133','8675-309J'],
        'count':['6','7','10','2','1']
        }
        )

jcrdf = pd.DataFrame(
        {
        'jobtitle': ['manager','technician','noob','retiree'],
        'location': ['loc1','loc3','loc4','loc2'],
        'jcode' : ['4444-4444','3333-3333','2222-2222','1111-1111'],
        'All_codes': ['0096-0225,0096-0225','1820-7448,0567-8315,0567-8315','0007-4977,0007-4977','0007-0963,0007-0963,0366-077X,1365-2133']                        
         })

I have a lookup that allows for a diff:

jcrdf_lookup = pd.DataFrame(jcrdf['All_codes'].str.split(',').tolist(),
                            index=jcrdf.jcode).stack(level=0).reset_index(level=0)
matches = jcrdf_lookup[jcrdf_lookup[0].isin(pmdf.code)]
jcrdfmatch = jcrdf[jcrdf.jcode.isin(matches.jcode)]
jcrdfnomatch = pmdf[~pmdf.code.isin(matches[0])]

But I can't figure out how to include pmdf.count.

I tried making a df of the unique codes from matches, but no matter what those values have to be in jcfdf.All_codes.

Thanks in advance, as usual, for any assistance.

One way would be to expand jcrdf All_codes column and then use merge

jcrdf_temp = jcrdf.set_index(['jcode', 'jobtitle', 'location']).All_codes.str.split(',',expand = True)\
.stack().reset_index(3,drop = True).reset_index(name = 'All_codes')

new_df = pd.merge(pmdf, jcrdf_temp, left_on = 'code', right_on = 'All_codes')

You get

    code    count   jcode       jobtitle    location    All_codes
0   0567-8315   6   3333-3333   technician  loc3    0567-8315
1   0567-8315   6   3333-3333   technician  loc3    0567-8315
2   0007-4977   7   2222-2222   noob        loc4    0007-4977
3   0007-4977   7   2222-2222   noob        loc4    0007-4977
4   0096-0225   10  4444-4444   manager     loc1    0096-0225
5   0096-0225   10  4444-4444   manager     loc1    0096-0225
6   1365-2133   2   1111-1111   retiree     loc2    1365-2133

If you want the data in original format

new_df = new_df.drop('All_codes', 1).groupby(['jcode', 'jobtitle', 'count', 'location']).code.apply(','.join).reset_index()

    jcode       jobtitle    count   location    code
0   1111-1111   retiree     2       loc2        1365-2133
1   2222-2222   noob        7       loc4        0007-4977,0007-4977
2   3333-3333   technician  6       loc3        0567-8315,0567-8315
3   4444-4444   manager     10      loc1        0096-0225,0096-0225

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM