如何遍历 Python 数据框，偶尔需要就地用一行替换两行？

Question

The "in-place" is the aspect I struggle with- creating a brand new data frame is solved (provided at the end). My specific issue is that my imported data occasionally splits one column's string into two substrings, placing the first substring on one row with its other columns of data and placing the second substring on the following row with NaN values for its other columns.

This is what the data frame should look like:

            Actor   Color  Number
0       Amy Adams     red       1
1       Bill Burr  orange       2
2    Courtney Cox  yellow       3
3    Danny DeVito   green       4
4  Emilio Estevez    blue       5

This is what my imported data frame initially looks like, where "Courtney Cox" and "Emilio Estevez" have been split into two rows. I provided the code to create this data frame. (Don't worry about the shift from integer to float- it's irrelevant.)

          Actor   Color  Number
0     Amy Adams     red     1.0
1     Bill Burr  orange     2.0
2      Courtney  yellow     3.0
3           Cox     NaN     NaN
4  Danny DeVito   green     4.0
5        Emilio    blue     5.0
6       Estevez     NaN     NaN

bad_df = pd.DataFrame({'Actor': ['Amy Adams','Bill Burr','Courtney','Cox','Danny DeVito','Emilio','Estevez'],
                       'Color':['red','orange','yellow',np.nan,'green','blue',np.nan],
                       'Number':[1,2,3,np.nan,4,5,np.nan]})

I do have access to the correct list for the Actor column.

actor_list = ['Amy Adams','Bill Burr','Courtney Cox','Danny DeVito','Emilio Estevez']

My data frames are actually pretty small, so copying the data frame or creating a separate data frame isn't a problem, but it seems like I should be able to perform my fix in-place.

Here's my current approach (iteratively creating a new data frame), but it seems sloppy. I iterate through a zip where each element consists of the index of a row, the row's Actor string, and the next row's Actor string. However, I have to do the last row outside of the loop so I don't look for a "next row" that doesn't exist.

new_df = pd.DataFrame()
for a1idx, a1, a2 in zip(bad_df.iloc[:-1,0].index, bad_df.iloc[:-1,0], bad_df.iloc[1:,0]):
    if a1 in actor_list: # First and last name are in this row
        new_df = new_df.append(bad_df.iloc[a1idx,:]) # Add row
    elif a1 + ' ' + a2 in actor_list: # First and last name are in consecutive rows
        new_df = new_df.append(bad_df.iloc[a1idx,:]) # Add row
        new_df.iloc[-1,0] = a1 + ' ' + a2 # Correct name in row
    # If neither of the above if conditions are met, this means we're inefficiently
    # looking at a row with just a last name which was dealt with in the previous iteration
if bad_df.iloc[-1,0] in actor_list: # Check very last row of data frame
    new_df = new_df.append(bad_df.iloc[-1,:]) # Add row

Is there a way to do this in-place?

Answer 1

Would that be a better way?

import pandas as pd

bad_df = pd.DataFrame({'Actor': ['Amy Adams','Bill Burr','Courtney','Cox','Danny DeVito','Emilio','Estevez'],
                       'Color':['red','orange','yellow',np.nan,'green','blue',np.nan],
                       'Number':[1,2,3,np.nan,4,5,np.nan]})

actor_list = ['Amy Adams','Bill Burr','Courtney Cox','Danny DeVito','Emilio Estevez']

nan_index = bad_df['Color'].isna()
bad_df.loc[nan_index, 'last_names'] = bad_df['Actor'][nan_index]
bad_df['last_names'] = bad_df['last_names'].shift(-1)
mask = pd.Series(nan_index).shift(-1, fill_value=False)
bad_df.loc[mask, 'Actor'] = bad_df['Actor'].str.cat(bad_df['last_names'], sep=' ')
bad_df.drop('last_names', axis=1, inplace=True)
bad_df = bad_df[~nan_index]

print(bad_df)

Output:

            Actor   Color  Number
0       Amy Adams     red     1.0
1       Bill Burr  orange     2.0
2    Courtney Cox  yellow     3.0
4    Danny DeVito   green     4.0
5  Emilio Estevez    blue     5.0