Pandas Dataframe-在一个数据框中不存在一列时合并数据

Question

There are two sets of csv files. File type1 has following columns:

col1, node, id, col4...... col100, dest.
ABC, 1, 1000, XY, ..... ax, LA
XYZ, 3, 3000, TY, ......ty, NY
WAR, 2, 2000, MJ, ......rr, London

File type2 has following columns:

col101, node-name, col102, col103..... col200, dest
ark, 16, ty, tuu, ...., bfg, Mumbai
raid,25, by, why, ...., cgh, Nairobi

My requirement is the following: Create a file that contains id, node/node-name and dest. id is not available in the second file and so must be noted as 0 corresponding to the node-name entries. So the data would look like

1000, 1, LA
2000, 2, London
3000, 3, NY
0, 16, Mumbai
0, 25, Nairobi

This is the code that I am writing.

# frames contains all the files and their data as filename and as a data frame.
for fileName, frame in frames.items():
  nodeinfo = frame.columns.values.tolist()[1]
  if nodeinfo == 'node-name':
    entry = 'node-name'
  else:
    entry = 'node'
  if entry in frame:
    frame1= frame[[entry, 'dest']]
    if 'id' in frame:
      idFrame = frame[['id', entry]]
    mergeFrame = pandas.merge(idFrame, frame1, how = 'right', on = entry)
    uniqFrame = mergeFrame.drop_duplicates([entry])

Of course there is a problem with the logic that mergeFrame may throw an error if the idFrame does not exist because there is no 'id' in the file. Honestly, I am a bit lost. Any help is appreciated.

Thank you, Anoop

Answer 1

You can add a dummy column in file2 DF like below:

frame2['id'] = 0

This will add a new column id in frame2 with values 0 for all rows. Now you have consistent columns in both frames.

Rename node-name column to node in Frame2.

frame2.rename(columns={'node-name':'node'}, inplace=True)

Then, you can use concat :

pd.concat([frame1[['id','node','dest']], frame2[['id','node','dest']]])