Converting python list to pandas dataframe selecting specific strings from the list

Question

I have the below python list:

w=[[['I=427', 'PLAN=1'], 'A=0PDB'],
 [['I=427', 'PLAN=1'], 'B=40NGC'],
 [['I=427', 'PLAN=1'], 'C=21#NGA'],
 [['I=429', 'PLAN=1'], 'A=0PDB'],
 [['I=429', 'PLAN=1'], 'B=18C'],
 [['I=429', 'PLAN=1'], 'C=28TGD'],
 [['I=429', 'PLAN=1'], 'D=18TGA'],
 [['I=429', 'PLAN=1'], 'E=1A'],
 [['I=429', 'PLAN=2'], 'A=0PDB'],
 [['I=429', 'PLAN=2'], 'B=17C']]

How can I convert it to the below pandas DataFrame:

So, from the second string in the list I want to select the first string, the number after equal sign and the last string. For example in B=40NGC , I want to choose B , 40 , C and put it into the DataFrame.

Answer 1

Here's one approach:

Rework w a bit to create a list of lists and build a DataFrame. Then extract a digits from green_time column:

out = []
for lst, s in w:
    phase, rest = s.split('=')
    green_time, next_phase = rest[:-1], rest[-1]
    out.append(lst + [phase, green_time, next_phase])
out = pd.DataFrame(out, columns=['site_no', 'plan', 'phase', 'green_time','next_phase'])
out['green_time'] = out['green_time'].str.extract('(\d+)')

Alternatively, we could pass w to the DataFrame constructor and use str.extract to extract the relevant items in columns:

df = pd.DataFrame(w)
df = df.join(pd.DataFrame(df[0].tolist(), columns=['site_no', 'plan']))
df[['phase', 'green_time','next_phase']] = df[1].str.extract('(\w)=(\d+)([^0-9]+)')
df['next_phase'] = df['next_phase'].str[-1]
df = df.drop(columns=[0,1])

Output:

  site_no    plan phase green_time next_phase
0   I=427  PLAN=1     A          0          B
1   I=427  PLAN=1     B         40          C
2   I=427  PLAN=1     C         21          A
3   I=429  PLAN=1     A          0          B
4   I=429  PLAN=1     B         18          C
5   I=429  PLAN=1     C         28          D
6   I=429  PLAN=1     D         18          A
7   I=429  PLAN=1     E          1          A
8   I=429  PLAN=2     A          0          B
9   I=429  PLAN=2     B         17          C