python从数据集中删除特殊值

Question

I have a large data set of movies. I am using Pandas package.

In the variable "budget" (is an object type) there is often the "?" charachter.

Now I want to remove all movies, that contain a "?" in the budget variable.

In the end I want to convert the budget variable into a integer and run a regression with the variable "quality". Picture of the data set I have tried one method, but it doesn´t worked out. while "?" in df.budget: df.remove(?)

Answer 1

Compare for not equal ( != ) numpy array created by values or convert all values to string s first and add all for check all True s values per rows, last filter by boolean indexing :

df = pd.DataFrame({'A':list('abcde?'),
                   'B':[4,5,4,5,5,4],
                   'C':[7,8,9,4,2,3],
                   'D':[4,3,5,7,1,0],
                   'E':[5,3,6,9,2,4],
                   'F':list('aaabbb')})

print (df)
   A  B  C  D  E  F
0  a  4  7  4  5  a
1  b  5  8  3  3  a
2  c  4  9  5  6  a
3  d  5  4  7  9  b
4  e  5  2  1  2  b
5  ?  4  3  0  4  b

df = df[(df.values != '?').all(axis=1)]
#alternative
#df = df[(df.astype(str) != '?').all(axis=1)]
print (df)
   A  B  C  D  E  F
0  a  4  7  4  5  a
1  b  5  8  3  3  a
2  c  4  9  5  6  a
3  d  5  4  7  9  b
4  e  5  2  1  2  b

Details :

print (df.values != '?')
[[ True  True  True  True  True  True]
 [ True  True  True  True  True  True]
 [ True  True  True  True  True  True]
 [ True  True  True  True  True  True]
 [ True  True  True  True  True  True]
 [False  True  True  True  True  True]]

print ((df.values != '?').all(axis=1))
[ True  True  True  True  True False]

EDIT:

df = pd.DataFrame({'A':list('abcde?'),
                   'B':[4,5,4,5,5,4],
                   'C':[7,8,9,4,2,3],
                   'D':[4,3,5,7,1,'?'],
                   'E':['?',3,6,'?',2,4]}).astype(str)

print (df)
   A  B  C  D  E
0  a  4  7  4  ?
1  b  5  8  3  3
2  c  4  9  5  6
3  d  5  4  7  ?
4  e  5  2  1  2
5  ?  4  3  ?  4

#replace only in columns from list
cols = ['C','D','E']
#if only ? with numeric to NaNs
df[cols] = df[cols].replace('?', np.nan).astype(float)

#replace all non numeric to NaNs
#df[cols] = df[cols].apply(lambda x: pd.to_numeric(x, errors='coerce'))

#replace NaNs by means
df = df.fillna(df.mean())
print (df)
   A  B    C    D     E
0  a  4  7.0  4.0  3.75
1  b  5  8.0  3.0  3.00
2  c  4  9.0  5.0  6.00
3  d  5  4.0  7.0  3.75
4  e  5  2.0  1.0  2.00
5  ?  4  3.0  4.0  4.00