How to read from a csv file and add the values as headers of pandas dataframe in python?

Question

I have a csv file which contains a list of headers which needs to be passed as headers to a target dataframe

csv file contains this :

DeliveryDate,DeliveryTimestamp,event_comments

input dataframe looks like this :

DeliveryDate  DeliveryTimestamp
1603479130     1260347913
45603479130    21260347913

the target dataframe should look like this :

DeliveryDate  DeliveryTimestamp   event_comments
1603479130     1260347913         nan
45603479130    21260347913        nan

I am trying to read the headers from a csv file and pass as dataframe columns:

with open(base_path+'/'+'final_headers.csv', newline='') as f:
        reader = csv.reader(f)
        data = list(reader)
consolidated_df_cleaner.columns=data
consolidated_df_cleaner.to_csv(base_path +'/'+'target.csv', index=False)

it shows this error :

ValueError: Length mismatch: Expected axis has 22 elements, new values have 3 elements

Answer 1

Instead, you can create an empty dataframe first using the csv-header-file:

csv_df = pd.read_csv(csv_filename)
newcolumns = set(csv_df.columns) - set(input_df.columns)

for ncol in newcolumns:
  input_df[ncol] = np.nan