Key Error: None of [Int64Index…] dtype='int64] are in the [columns]

Question

I'm trying to run k-fold cross validation on pipeline(Standardscaler,DecisionTreeClassifier).

First, I import the data.

data = pd.read_csv('train_strokes.csv')

Then preprocessing dataframe

# Preprocessing data 
data.drop('id',axis=1,inplace=True)
data['age'] =data['age'].apply(lambda x : x if round(x) else np.nan) 
data['bmi'] = data['bmi'].apply(lambda bmi : bmi if 12< bmi <45 else np.nan)
data['gender'] = data['gender'].apply(lambda gender : gender if gender =='Female' or gender =='Male' else np.nan)
data.sort_values(['gender', 'age','bmi'], inplace=True) 
data['bmi'].ffill(inplace=True)
data.dropna(axis=0,inplace=True)
data.reset_index(drop=True, inplace=True)

#categorial data to numeric value
enc = LabelEncoder()
data['gender'] = enc.fit_transform(data['gender'])
data['work_type'] = enc.fit_transform(data['work_type'])
data['Residence_type'] = enc.fit_transform(data['Residence_type'])
data['smoking_status'] = enc.fit_transform(data['smoking_status'])
data['ever_married'] = enc.fit_transform(data['ever_married'])

then slice feature and target

target = data['stroke']
feat = data.drop('stroke',axis=1)

and Using SMOTE to balance the Data

sm = SMOTE(random_state = 1) 
feat, target = sm.fit_resample(feat, target) 
feat['age'] = feat['age'].apply(lambda x : round(x))
feat['hypertension'] = feat['hypertension'].apply(lambda x : round(x))
feat['heart_disease'] = feat['heart_disease'].apply(lambda x : round(x))
feat['ever_married'] = feat['ever_married'].apply(lambda x : round(x))
#split training and test
X_train, X_test, y_train, y_test = train_test_split(feat, target, test_size=0.3, random_state= 2)

It's part of the problem.

Kfold =KFold(n_splits=10)
pipeline = make_pipeline(StandardScaler(), DecisionTreeClassifier())
n_iter = 0
for train_idx, test_idx in Kfold.split(feat):
    pipeline.fit(X_train[train_idx], y_train[train_idx])
    score = pipeline.score(X_train[test_idx],y_train[test_idx])
    print('Fold #{} accuracy{}'.format(1,score))

ERROR CODE

Traceback (most recent call last):
File "/Users/merb/Documents/Dev/DataScience/TP.py", line 84, in <module>
pipeline.fit(X_train[train_idx], y_train[train_idx])
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site- 
packages/pandas/core/frame.py", line 3030, in __getitem__
indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-  
packages/pandas/core/indexing.py", line 1266, in _get_listlike_indexer
self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-   
packages/pandas/core/indexing.py", line 1308, in _validate_read_indexer
raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([ 5893,  5894,  5895,  5896,  5897,  5898,  5899,  5900,    
5901,\n             5902,\n            ...\n            58912, 58913, 58914, 58915, 
58916, 58917, 58918, 58919, 58920,\n            58921],\n           dtype='int64', 
length=53029)] are in the [columns]"

Answer 1

You should use df.loc[indexes] to select rows by their indexes. If you want to select rows by their integer location you should use df.iloc[indexes] .

In addition to that, you can read this page on Indexing and Selecting data with pandas.