Python，Pandas Dataframe - 创建带有条件语句的派生列

Question

I have a Pandas DataFrame with 50 columns and 50k rows. There is one column with measurement data that needs correcting with a calibration factor. The factor is an integer value to be added or substracted. There are multiple (10ish) measurements in the same column of measurement data ['T_calibrated'], they all have an unique serial number in a seperate column ['serial']

I can calibrate a single sensor as follows using .where:

data['T_calibrated'] = data['T_uncalibrated'].where(data['serial'] == 12345)-2.7

12345 is the unique serial number -2.7 is the calibration factor.

How would I write this in a more generic form so that I can add the unique calibration factor associated with each serial number and add this all as a single combined column ['T_calibrated']. So far I'm getting stuck with brute force ways. I'm sure there must be some very elegant way to do this.

I have a second dataframe with the serial number and calibration factor that can be looped or compared with ofcourse.

Answer 1

Close after posting my question I saw the light.

I joined the two dataframes on the serial numbers preserving the original index of the original (because I want that). Then I created another column just subtracting the two values. I didn't know how to add "inplace=True" with the join statement.

Here's my code:

calibrated_data=data.join(calibration_dataframe.set_index('serial'),on='serial')
calibrated_data['T_calibrated'] = calibrated_data.T_uncalibrated - calibrated_data.calibration_factor

Answer 2

You describe two data frames structured as below. Simplest approach is to merge them then calculate required column from merged data frame.

import numpy as np

serial = [f"{a}{ord(a)}" for a in list("abcdef")]

df = pd.DataFrame({"serial":np.random.choice(serial, 50), "T_uncalibrated":np.random.randint(20,30,50)})
dfs = pd.DataFrame({"serial":serial, "calibration":np.random.randint(-2,2,len(serial))})

df.merge(dfs, on="serial").assign(T_calibrated=lambda d: d["T_uncalibrated"]+d["calibration"])

sample output

serial	T_uncalibrated	calibration	T_calibrated
c99	20	-2	18
c99	27	-2	25
c99	28	-2	26
c99	28	-2	26
c99	20	-2	18
c99	22	-2	20
c99	24	-2	22
c99	24	-2	22
d100	21	-1	20
d100	26	-1	25