当同一行中的数据添加到另一列时，计算一行的列值

Question

I have a large pandas data frame in python. I have seven columns of raw data that get updated all at once on a periodic basis, and I need to update the values in the new rows of the 84 other columns every time new data is added to the bottom of columns 1-7. I would like to do this without having to recalculate all the values of the entire 84 other columns. as there are millions of rows in these columns.

Answer 1

After doing the first calculation on the main dataframe, try doing the calculation for new data separately then concat them at the end (provided both have the same columns before concatenation).

import pandas as pd

columns = ['c1','c2','c3','c4','c5','c6','c7']

main = pd.read_csv('file.csv', names=columns)
# ... do your calculation

new = pd.read_csv('new_file.csv', names=columns)
# ... do your calculation

all = pd.concat([main, new])

# if you need to reset the index, use the following line instead:
# all = pd.concat([main, new], ignore_index=True)