Pandas Dataframe - 无循环的组合和子组合编号系统

Question

I'm working with Pandas and I have a big part list with Main Assy, Sub Assy I, Sub Assy II and Sub Assy III. Only one "Assy" column per row can be filled with a string in the dataframe. The aim is to transfer the arrangement of the parts into a numbering system.The following table shows the expected outcome :

Main Assy   Sub Assy I  Sub Assy II Sub Assy III    Level I Level II    Level III   Level IV
asd                                                    1        0            0         0
               fgd                                     1        1            0         0
                           sdd                         1        1            1         0
                           dsd                         1        1            2         0
                           fhg                         1        1            3         0
                                        tdc            1        1            3         1
                                        dyx            1        1            3         2
                                        dsg            1        1            3         3
               dfg                                     1        2            0         0
                           cvf                         1        2            1         0
                           ngs                         1        2            2         0
                           vbn                         1        2            3         0
                                        dsd            1        2            3         1
                                        vcd            1        2            3         2
                                        cbn            1        2            3         3
ged                                                    2        0            0         0
               dfs                                     2        1            0         0
                           aef                         2        1            1         0

My plan was to cumulate over the rows in the "Level"-columns as long as there are no changes in the higher level. When there is a change thus a new number on the higher level, the selected cell on a lower level needs to go back to zero. Is there no change it keeps the same number. I tried the following:

df[lambda df: df.columns[0:4]] = df[lambda df: df.columns[0:4]].isna()

for index in range(0,4):
    mask = ((df.iloc[:,index] == False))
    print(mask)
    df.iloc[:,(index+4)] = mask.groupby((~mask).cumsum()).cumsum().astype(int)

So I check if the cell is filled by searching for missing values. I don't want to use a loop with lots of conditions for every row because of a big data frame. I only used this one FOR-loop over the columns and tried to cumulate by creating a mask which shows changes from FALSE to TRUE.

The actual outcome is:

Main Assy   Sub Assy I  Sub Assy II Sub Assy III    Level I Level II    Level III   Level IV
asd                                                    1        0            0         0
               fgd                                     0        1            0         0
                           sdd                         0        0            1         0
                           dsd                         0        0            2         0
                           fhg                         0        0            3         0
                                        tdc            0        0            0         1
                                        dyx            0        0            0         2
                                        dsg            0        0            0         3
               dfg                                     0        2            0         0
                           cvf                         0        0            1         0
                           ngs                         0        0            2         0
                           vbn                         0        0            3         0
                                        dsd            0        0            0         1
                                        vcd            0        0            0         2
                                        cbn            0        0            0         3
ged                                                    2        0            0         0
               dfs                                     0        1            0         0
                           aef                         0        0            1         0

What would be the right way to setup the mentioned conditional counting without using loops?

Answer 1

Key

The change of output to be applied over each row can be fully determined by the current "level" and the previous level. Here "level" means the index number of the column having a non-zero entry.

In other words, a state variable retaining the level of the previous row is sufficient for populating the current row correctly.

Code

# the working dataset
df2 = df.iloc[:, :4].reset_index(drop=True)  # make a copy
df2.columns = range(4)  # rename columns to (0,1,2,3) for convenience

# output container
arr = np.zeros(df2.shape, dtype=int) 

# state variable: level of the last row
last_lv = 0

for idx, row in df2.iterrows():

    # get current indentation level
    lv = row.first_valid_index()

    if idx > 0:

        # case 1: same or decreased level
        if lv <= last_lv:
            # keep previous levels except current level
            arr[idx, :lv] = arr[idx-1, :lv]
            # current level++
            arr[idx, lv] = arr[idx-1, lv] + 1

        # case 2: increased level
        elif lv > last_lv:
            # keep previous levels
            arr[idx, :last_lv+1] = arr[idx - 1, :last_lv+1]
            # start counting the new levels
            arr[idx, last_lv+1:lv+1] = 1  

    # the first row
    else:
        arr[0, 0] = 1

    # update state variable for next use
    last_lv = lv

# append result to dataframe
df[["Level I", "Level II", "Level III", "Level IV"]] = arr

Result

print(df[["Level I", "Level II", "Level III", "Level IV"]])

    Level I  Level II  Level III  Level IV
0         1         0          0         0
1         1         1          0         0
2         1         1          1         0
3         1         1          2         0
4         1         1          3         0
5         1         1          3         1
6         1         1          3         2
7         1         1          3         3
8         1         2          0         0
9         1         2          1         0
10        1         2          2         0
11        1         2          3         0
12        1         2          3         1
13        1         2          3         2
14        1         2          3         3
15        2         0          0         0
16        2         1          0         0
17        2         1          1         0

Notes

1. The code just demonstrates what the logic looks like when progressing through each row. It is not quite optimized, so consider using more efficient representations of the data (eg numpy array or just a list of level numbers) when efficiency becomes a problem.
2. I have surveyed libraries for tree data structures such as anytree and treelib , hoping to find an automated way of outputting the tree hierarchy automagically. Unfortunately, I/O functions suitable for reading indented text files or comparable formats seemed to be lacking. This is the main reason why I decide to reinvent the wheel anyway.