如何将函数应用于pandas数据框中列的每个值？

Question

I had tried doing a somewhat manual approach using a loop like below:

data = pd.read_csv('data/training.csv')
for idx,imageString in enumerate(data.iloc[:,-1]):
    # print(imageString[0:10])
    data[idx,-1] = imageString.split(" ")

But this errors out on the last line with:

ValueError: Length of values does not match length of index

So my questions are:

1. Can anyone explain why I am getting the above error and how can I get around it?
2. Is this the proper way to apply a split to every value in the last column of my data frame?

Regarding #2 - I saw some people using applymap but I think this creates a new column, I really just want to replace the value in the existing column with another list.

Answer 1

I think you need str.split :

data = pd.read_csv('data/training.csv')
data.iloc[:,-1] = data.iloc[:,-1].str.split(expand=False)

Then select first or some another elements of lists with str[1] or str[n] :

data.iloc[:,-1] = data.iloc[:,-1].str.split(expand=False).str[0]
data.iloc[:,-1] = data.iloc[:,-1].str.split(expand=False).str[n]

Sample:

import pandas as pd

data = pd.DataFrame({'A':[1,2,3],
                   'B':[4,5,6],
                   'C':[7,8,9],
                   'D':[1,3,5],
                   'E':[5,3,6],
                   'F':['aa aa','ss uu','ee tt']})

print (data)
   A  B  C  D  E      F
0  1  4  7  1  5  aa aa
1  2  5  8  3  3  ss uu
2  3  6  9  5  6  ee tt

print (data.iloc[:,-1].str.split(expand=False))
0    [aa, aa]
1    [ss, uu]
2    [ee, tt]
Name: F, dtype: object

data.iloc[:,-1] = data.iloc[:,-1].str.split(expand=False).str[0]
print (data)
   A  B  C  D  E   F
0  1  4  7  1  5  aa
1  2  5  8  3  3  ss
2  3  6  9  5  6  ee

---

data.iloc[:,-1] = data.iloc[:,-1].str.split(expand=False).str[1]
print (data)
   A  B  C  D  E   F
0  1  4  7  1  5  aa
1  2  5  8  3  3  uu
2  3  6  9  5  6  tt

Can anyone explain why I am getting the above error and how can I get around it?

Problem is imageString.split(" ") return list and if assign to data[idx,-1] , length of elements of strings is less as length of all DataFrame.

Is this the proper way to apply a split to every value in the last column of my data frame?

Better is use string methods, see pandas documentation .

Answer 2

You are not accessing the values correctly.

To correct your code, the last line should be:

df.iat[idx, -1] = imageString.split(" ")

iat is used for scalar getting and setting.

This is probably a simpler way to accomplish your objective:

df.iloc[:, -1] = df.iloc[:, -1].str.split()