使用列名和行索引从 pandas dataframe 中选择值的正确方法是什么？

Question

what is the most efficient way of selecting value from pandas dataframe using column name and row index (by that I mean row number)?

I have a case where I have to iterate through rows:

I have a working solution:

i = 0
while i < len(dataset) -1:
    if dataset.target[i] == 1:
        dataset.sum_lost[i] = dataset['to_be_repaid_principal'][i] + dataset['to_be_repaid_interest'][i]
        dataset.ratio_lost[i] = dataset.sum_lost[i] / dataset['expected_returned_sum'][i]
    else:
        dataset.sum_lost[i] = 0
        dataset.ratio_lost[i]= 0
    i += 1   

But this solution is so much RAM hungry. I am also getting the following warning:

"A value is trying to be set on a copy of a slice from a DataFrame."

So I am trying to come up with another one:

i = 0
while i < len(dataset) -1:
    if dataset.iloc[i, :].loc['target'] == 1:
        dataset.iloc[i, :].loc['sum_lost'] = dataset.iloc[i, :].loc['to_be_repaid_principal'] + dataset.iloc[i, :].loc['to_be_repaid_interest']
        dataset.iloc[i, :].loc['ratio_lost'] = dataset.iloc[i, :].loc['sum_lost'] / dataset.iloc[i, :].loc['expected_returned_sum']
    else:
        dataset.iloc[i, :].loc['sum_lost'] = 0
        dataset.iloc[i, :].loc['ratio_lost'] = 0
    i += 1

But it does not work. I would like to come up with a faster/less ram hungry solution, because this will actually be web app a few users could use simultaneously.

Thanks a lot.

Answer 1

If you are thinking about "looping through rows", you are not using pandas right. You should think of terms of columns instead.

Use np.where which is vectorized (read: fast):

cond = dataset['target'] == 1
dataset['sumlost'] = np.where(cond, dataset['to_be_repaid_principal'] + dataset['to_be_repaid_interest'], 0)
dataset['ratio_lost'] = np.where(cond, dataset['sumlost'] / dataset['expected_returned_sum'], 0)