Replace python pandas df with values of a second dataframe based with condition

Question

I am new to python as I normally write scripts in R and therefore am learning to adjust to Pandas dataframes and nuances.

I have two lists of dicts that I turned into dataframes as I thought it would be easier to work with in that format.

df1= [{u'test': u'SAT Math', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': 404}, {u'test': u'SAT Verbal', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': 355}, {u'test': u'SAT Writing', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': 363}, {u'test': u'SAT Composite', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': 1122}, {u'test': u'ACT Math', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': None}, {u'test': u'ACT English', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': None}, {u'test': u'ACT Reading', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': None}, {u'test': u'ACT Science', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': None}, {u'test': u'ACT Composite', u'25th_percentile': None, u'75th_percentile': None, u'50th_percentile': None, u'mean': None}]


df2 = [{u'test': u'SAT Composite', u'mean': 1981}, {u'test': u'ACT Composite', u'mean': 29.6}]

I then put these as dataframes:

df1new = DataFrame(df1, columns=['test', '25th_percentile', 'mean', '50th_percentile','75th_percentile'])
df2new = DataFrame(df2)

Now, I would like to replace the contents of the column 'mean' in df1new if 'test' == "ACT Composite" and 'mean' is None

I have tried to use a combine_first approach, however I believe this requires the dataframes to be more similarly indexed. I have also tried:

if df1new['test'] == "ACT Composite" and df1new['mean'] == None:
            df1new['mean'] == df2new['mean']

as well as a .replace() variation.

Any advice would be greatly appreciated! Thank you in advance!

Answer 1

maybe this:

idx = (df1new.test == 'ACT Composite') & df1new['mean'].isnull()
df1new['mean'][idx] = df2new['mean'][1]

I added a [1] up there because i suppose that is what you want, the mean value corresponding to ACT Composite in df2new . it could also be written as

df1new['mean'][idx] = df2new['mean'][df2new.test == 'ACT Composite']