熊猫-如何拆分和合并名称相似的列？

Question

I have a dirty dataframe which needs column cleaning. Basically, there are a LOT of columns which contain combination data which should not be and slight spelling differences! For example:

         1    1/2    2c     2 c     
row
1       B     nan    C       nan 
2       B     nan    C       nan
3       nan   Rb     nan     nan
4       c     nan    nan     C

to something like this:

         1    2c    
row
1       B     C       
2       B     C       
3       Rb    Rb   
4       c     C

Thus the issue is two fold, how do you merge columns which are split on fuzzy logic similarity and how do you split then merge on columns which have combo values?

The only way I know how to do this would be to create a new column which uses the .apply function to apply if statements, but given that the number of columns is in the 100s this would be painful. Any ideas for a less manual solution?

Answer 1

Try

d0 = df.filter(regex='/')      # Grab the columns with "/" in name
d1 = df.drop(d0, 1)            # Drop those columns

a = d0.to_numpy()              
m = d0.columns.str.count('/')  # Count the number of "/".

d2 = pd.DataFrame(
    a.repeat(m + 1, axis=1),   # Repeat the columns one more time than the # of "/"
    d0.index,
    np.concatenate(d0.columns.str.split('/')) 
)

d3 = pd.concat([d1, d2], axis=1)  # Smash them back together

# Grab the first bit of the column name as long as they are digits
# Group by that and take the first non-null value
d3.groupby(np.ravel(d3.columns.str.extract('(\d+)')), axis=1).first()

    1   2
1   B   C
2   B   C
3  Rb  Rb
4   c   C