如何使用列之间的映射“重新排列”熊猫数据框？

Question

I'm new to pandas and data-frames so I hope this is not a too basic question.

Suppose I have (outer) merged two data frames and obtained the following:

     date_a  symbol_a  value_a  type_a symbol_b  date_b  value_b  type_b    total
0    yymmdd    AAAA       10       W    AAAA     yymmdd     2        S        12
1    yymmdd    BBBB        5       W    BBBB     yymmdd     2        S         7
2    yymmdd    CCCC       12       W     NaN        NaN     NaN     NaN      NaN
3    yymmdd    DDDD       15       W     NaN        NaN     NaN     NaN      NaN
4     NaN       NaN      NaN      NaN   EEEE     yymmdd      5       S       NaN
5     NaN       NaN      NaN      NaN   FFFF     yymmdd     10       S       NaN
6     NaN       NaN      NaN      NaN   GGGG     yymmdd      2       S       NaN

this is so far quite what I want (the dates are all the same day).

Now suppose I have a rule that tells me that I have a mapping (dictionary):

mapper = {'EEEE':'CCCC', 'FFFF':'DDDD'}

I may have symbol_b that are not mapped to any symbol_a (like index 6 above).

My question is, is it possible to use re-arrange the rows according to the mapping given by the dictionary so to get:

     date_a  symbol_a  value_a  type_a symbol_b  date_b  value_b  type_b    total
0    yymmdd    AAAA       10       W    AAAA     yymmdd     2        S        12
1    yymmdd    BBBB        5       W    BBBB     yymmdd     2        S         7
2    yymmdd    CCCC       12       W    EEEE     yymmdd     5        S        17
3    yymmdd    DDDD       15       W    FFFF     yymmdd     10       S        25
6     NaN       NaN      NaN      NaN   GGGG     yymmdd      2       S       NaN

Ideally then I would also like to get rid of all the rows that contain a NaN element, so that the final result would be:

     date_a  symbol_a  value_a  type_a symbol_b  date_b  value_b  type_b    total
0    yymmdd    AAAA       10       W    AAAA     yymmdd     2        S        12
1    yymmdd    BBBB        5       W    BBBB     yymmdd     2        S         7
2    yymmdd    CCCC       12       W    EEEE     yymmdd     5        S        17
3    yymmdd    DDDD       15       W    FFFF     yymmdd     10       S        25

Answer 1

So, I split your dataframe to the original 2 dataframes, just change the name of columns in dataframe to fit your usage:

import pandas as pd

a = pd.read_csv('test.csv')

#Split your merge df into 2 original df
del a['total']
df1 = a.loc[:, ['date_a', 'symbol_a', 'value_a', 'type_a']]
df2 = a.loc[:, ['date_b', 'symbol_b', 'value_b', 'type_b']]

df1.fillna('', inplace=True)
df2.fillna('', inplace=True)

df1 = df1[df1['date_a']!='']
df2 = df2[df2['date_b']!='']

#Add a mapping column to df2
mapper = {'EEEE':'CCCC', 'FFFF':'DDDD'}

df2['mapping'] = df2['symbol_b'].apply(lambda x: mapper.get(x) if mapper.get(x)!= None else x)

df1 = df1.merge(df2, left_on='symbol_a', right_on='mapping')
df1['total'] = df1['value_a'] + df1['value_b']
df1

The results in df1 is now:

    date_a  symbol_a    value_a type_a  date_b  symbol_b    value_b type_b  mapping total
0   yymmdd  AAAA        10.0    W       yymmdd  AAAA        2.0     S       AAAA    12.0
1   yymmdd  BBBB        5.0     W       yymmdd  BBBB        2.0     S       BBBB    7.0
2   yymmdd  CCCC        12.0    W       yymmdd  EEEE        5.0     S       CCCC    17.0
3   yymmdd  DDDD        15.0    W       yymmdd  FFFF        10.0    S       DDDD    25.0