[英]Key Error: "None of [Int64Index...] dtype='int64] are in the columns"
[英]Key Error: None of [Int64Index...] dtype='int64] are in the columns
我正在嘗試使用 np.random.shuffle() 方法洗牌我的索引,但我不斷收到一個我不明白的錯誤。 如果有人能幫我解決這個問題,我將不勝感激。 謝謝!
當我在開始時創建 raw_csv_data 變量時,我嘗試使用 delimiter=',' 和 delim_whitespace=0 ,因為我認為這是另一個問題的解決方案,但它一直拋出相同的錯誤
import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
#%%
raw_csv_data= pd.read_csv('Absenteeism-data.csv')
print(raw_csv_data)
#%%
df= raw_csv_data.copy()
print(display(df))
#%%
pd.options.display.max_columns=None
pd.options.display.max_rows=None
print(display(df))
#%%
print(df.info())
#%%
df=df.drop(['ID'], axis=1)
#%%
print(display(df.head()))
#%%
#Our goal is to see who is more likely to be absent. Let's define
#our targets from our dependent variable, Absenteeism Time in Hours
print(df['Absenteeism Time in Hours'])
print(df['Absenteeism Time in Hours'].median())
#%%
targets= np.where(df['Absenteeism Time in Hours']>df['Absenteeism Time
in Hours'].median(),1,0)
#%%
print(targets)
#%%
df['Excessive Absenteeism']= targets
#%%
print(df.head())
#%%
#Let's Separate the Day and Month Values to see if there is
correlation
#between Day of week/month with absence
print(type(df['Date'][0]))
#%%
df['Date']= pd.to_datetime(df['Date'], format='%d/%m/%Y')
#%%
print(df['Date'])
print(type(df['Date'][0]))
#%%
#Extracting the Month Value
print(df['Date'][0].month)
#%%
list_months=[]
print(list_months)
#%%
print(df.shape)
#%%
for i in range(df.shape[0]):
list_months.append(df['Date'][i].month)
#%%
print(list_months)
#%%
print(len(list_months))
#%%
#Let's Create a Month Value Column for df
df['Month Value']= list_months
#%%
print(df.head())
#%%
#Now let's extract the day of the week from date
df['Date'][699].weekday()
#%%
def date_to_weekday(date_value):
return date_value.weekday()
#%%
df['Day of the Week']= df['Date'].apply(date_to_weekday)
#%%
print(df.head())
#%%
df= df.drop(['Date'], axis=1)
#%%
print(df.columns.values)
#%%
reordered_columns= ['Reason for Absence', 'Month Value','Day of the
Week','Transportation Expense', 'Distance to Work', 'Age',
'Daily Work Load Average', 'Body Mass Index', 'Education',
'Children',
'Pets',
'Absenteeism Time in Hours', 'Excessive Absenteeism']
#%%
df=df[reordered_columns]
print(df.head())
#%%
#First Checkpoint
df_date_mod= df.copy()
#%%
print(df_date_mod)
#%%
#Let's Standardize our inputs, ignoring the Reasons and Education
Columns
#Because they are labelled by a separate categorical criteria, not
numerically
print(df_date_mod.columns.values)
#%%
unscaled_inputs= df_date_mod.loc[:, ['Month Value','Day of the
Week','Transportation Expense','Distance to Work','Age','Daily Work
Load
Average','Body Mass Index','Children','Pets','Absenteeism Time in
Hours']]
#%%
print(display(unscaled_inputs))
#%%
absenteeism_scaler= StandardScaler()
#%%
absenteeism_scaler.fit(unscaled_inputs)
#%%
scaled_inputs= absenteeism_scaler.transform(unscaled_inputs)
#%%
print(display(scaled_inputs))
#%%
print(scaled_inputs.shape)
#%%
scaled_inputs= pd.DataFrame(scaled_inputs, columns=['Month Value','Day
of the Week','Transportation Expense','Distance to Work','Age','Daily
Work Load Average','Body Mass Index','Children','Pets','Absenteeism
Time
in Hours'])
print(display(scaled_inputs))
#%%
df_date_mod= df_date_mod.drop(['Month Value','Day of the
Week','Transportation Expense','Distance to Work','Age','Daily Work
Load Average','Body Mass Index','Children','Pets','Absenteeism Time in
Hours'], axis=1)
print(display(df_date_mod))
#%%
df_date_mod=pd.concat([df_date_mod,scaled_inputs], axis=1)
print(display(df_date_mod))
#%%
df_date_mod= df_date_mod[reordered_columns]
print(display(df_date_mod.head()))
#%%
#Checkpoint
df_date_scale_mod= df_date_mod.copy()
print(display(df_date_scale_mod.head()))
#%%
#Let's Analyze the Reason for Absence Category
print(df_date_scale_mod['Reason for Absence'])
#%%
print(df_date_scale_mod['Reason for Absence'].min())
print(df_date_scale_mod['Reason for Absence'].max())
#%%
print(df_date_scale_mod['Reason for Absence'].unique())
#%%
print(len(df_date_scale_mod['Reason for Absence'].unique()))
#%%
print(sorted(df['Reason for Absence'].unique()))
#%%
reason_columns= pd.get_dummies(df['Reason for Absence'])
print(reason_columns)
#%%
reason_columns['check']= reason_columns.sum(axis=1)
print(reason_columns)
#%%
print(reason_columns['check'].sum(axis=0))
#%%
print(reason_columns['check'].unique())
#%%
reason_columns=reason_columns.drop(['check'], axis=1)
print(reason_columns)
#%%
reason_columns=pd.get_dummies(df_date_scale_mod['Reason for Absence'],
drop_first=True)
print(reason_columns)
#%%
print(df_date_scale_mod.columns.values)
#%%
print(reason_columns.columns.values)
#%%
df_date_scale_mod= df_date_scale_mod.drop(['Reason for Absence'],
axis=1)
print(df_date_scale_mod)
#%%
reason_type_1= reason_columns.loc[:, 1:14].max(axis=1)
reason_type_2= reason_columns.loc[:, 15:17].max(axis=1)
reason_type_3= reason_columns.loc[:, 18:21].max(axis=1)
reason_type_4= reason_columns.loc[:, 22:].max(axis=1)
#%%
print(reason_type_1)
print(reason_type_2)
print(reason_type_3)
print(reason_type_4)
#%%
print(df_date_scale_mod.head())
#%%
df_date_scale_mod= pd.concat([df_date_scale_mod,
reason_type_1,reason_type_2, reason_type_3, reason_type_4], axis=1)
print(df_date_scale_mod.head())
#%%
print(df_date_scale_mod.columns.values)
#%%
column_names= ['Month Value','Day of the Week','Transportation
Expense',
'Distance to Work','Age','Daily Work Load Average','Body Mass Index',
'Education','Children','Pets','Absenteeism Time in Hours',
'Excessive Absenteeism', 'Reason_1', 'Reason_2', 'Reason_3',
'Reason_4']
df_date_scale_mod.columns= column_names
print(df_date_scale_mod.head())
#%%
column_names_reordered= ['Reason_1', 'Reason_2', 'Reason_3',
'Reason_4','Month Value','Day of the Week','Transportation Expense',
'Distance to Work','Age','Daily Work Load Average','Body Mass Index',
'Education','Children','Pets','Absenteeism Time in Hours',
'Excessive Absenteeism']
df_date_scale_mod=df_date_scale_mod[column_names_reordered]
print(display(df_date_scale_mod.head()))
#%%
#Checkpoint
df_date_scale_mod_reas= df_date_scale_mod.copy()
print(df_date_scale_mod_reas.head())
#%%
#Let's Look at the Education column now
print(df_date_scale_mod_reas['Education'].unique())
#This shows us that education is rated from 1-4 based on level
#of completion
#%%
print(df_date_scale_mod_reas['Education'].value_counts())
#The overwhelming majority of workers are highschool educated, while
the
#rest have higher degrees
#%%
#We'll create our dummy variables as highschool and higher education
df_date_scale_mod_reas['Education']=
df_date_scale_mod_reas['Education'].map({1:0, 2:1, 3:1, 4:1})
#%%
print(df_date_scale_mod_reas['Education'].unique())
#%%
print(df_date_scale_mod_reas['Education'].value_counts())
#%%
#Checkpoint
df_preprocessed= df_date_scale_mod_reas.copy()
print(display(df_preprocessed.head()))
#%%
#%%
#Split Inputs from targets
scaled_inputs_all= df_preprocessed.loc[:,'Reason_1':'Absenteeism Time
in
Hours']
print(display(scaled_inputs_all.head()))
print(scaled_inputs_all.shape)
#%%
targets_all= df_preprocessed.loc[:,'Excessive Absenteeism']
print(display(targets_all.head()))
print(targets_all.shape)
#%%
#Shuffle Inputs and targets
shuffled_indices= np.arange(scaled_inputs_all.shape[0])
np.random.shuffle(shuffled_indices)
shuffled_inputs= scaled_inputs_all[shuffled_indices]
shuffled_targets= targets_all[shuffled_indices]
這是我嘗試洗牌時不斷遇到的錯誤:
KeyError Traceback (most recent call last) in 1 shuffled_indices= np.arange(scaled_inputs_all.shape[0]) 2 np.random.shuffle(shuffled_indices) ----> 3 shuffled_inputs= scaled_inputs_all[shuffled_indices] 4 shuffled_targets= targets_all[shuffled_indices]
~\Anaconda3\lib\site-packages\pandas\core\frame.py in getitem (self, key) 2932 key = list(key) 2933 indexer = self.loc._convert_to_indexer(key, axis=1, -> 2934 raise_missing =True) 2935 2936 # take() 不接受布爾索引器
~\Anaconda3\lib\site-packages\pandas\core\indexing.py in _convert_to_indexer(self, obj, axis, is_setter, raise_missing) 1352 kwargs = {'raise_missing': True if is_setter else 1353
raise_missing} -> 1354 return self._get_listlike_indexer(obj, axis, **kwargs)[1] 1355 else: 1356 try:~\Anaconda3\lib\site-packages\pandas\core\indexing.py in _get_listlike_indexer(self, key, axis, raise_missing) 1159 self._validate_read_indexer(keyarr, indexer, 1160
o._get_axis_number(axis), -> 1161 raise_missing=raise_missing) 1162 return keyarr, indexer
1163~\Anaconda3\lib\site-packages\pandas\core\indexing.py 在 _validate_read_indexer(self,key,indexer,axis,raise_missing)1244 raise KeyError(1245
u"None of [{key}] are in the [{axis}]".format( -> 1246 key=key, axis=self.obj._get_axis_name(axis))) 1247 1248 # 我們(暫時)允許一些缺少帶有 .loc 的鍵,但在KeyError:“沒有 [Int64Index([560, 320, 405, 141, 154, 370, 656, 26, 444, 307,\n ...\n 429, 542, 676, 588, 315, 284, 293, 607, 197, 250],\n dtype='int64', length=700)] 在 [columns]"
您使用loc
函數創建了scaled_inputs_all
DataFrame,因此它很可能不包含連續索引。
另一方面,您將shuffled_indices
創建為僅從一系列連續數字中進行的隨機播放。
請記住scaled_inputs_all[shuffled_indices]
獲取scaled_inputs_all
的行,其索引值等於shuffled_indices
的元素。
也許你應該寫:
scaled_inputs_all.iloc[shuffled_indices]
請注意, iloc
提供基於整數位置的索引,而不管索引值如何,即正是您需要的。
可能有人在機器學習中使用 KFOLD 時也會遇到同樣的錯誤。
解決方案如下:
您需要使用 iloc:
X_train, X_test = X.iloc[train_index], X.iloc[test_index]
y_train, y_test = y.iloc[train_index], y.iloc[test_index]
我也有這個問題。 我通過將數據框和系列更改為數組來解決它。
嘗試以下代碼行:
scaled_inputs_all.iloc[shuffled_indices].values
如果您在從數據框中刪除行后重置索引,這應該會停止關鍵錯誤。
您可以在運行df.drop
后運行它來做到這一點:
df = df.reset_index(drop=True)
或者,等效地:
df.reset_index(drop=True, inplace=True)
根據列值條件刪除具有索引的行時發生以下錯誤:
return self._engine.get_loc(key) 文件“pandas/_libs/index.pyx”,第 107 行,在 pandas._libs.index.IndexEngine.get_loc 文件“pandas/_libs/index.pyx”,第 131 行,在 pandas。 _libs.index.IndexEngine.get_loc 文件“pandas/_libs/hashtable_class_helper.pxi”,第 992 行,在 pandas._libs.hashtable.Int64HashTable.get_item 文件“pandas/_libs/hashtable_class_helper.pxi”,第 998 行,在 pandas._libs。 hashtable.Int64HashTable.get_item KeyError:226
在處理上述異常的過程中,又出現了一個異常:
回溯(最近一次通話最后):
要解決此問題,請創建一個索引列表並立即刪除行,如下所示:
df.drop(index=list1,labels=None, axis=0, inplace=True,columns=None, level=None, errors='raise')
對於 Kfold 拆分,您需要 iloc ......我用過這個。
for train_i, test_i in kf.split(X):
print("TRAIN:", train_i, "TEST:", test_i)
X_train, X_test = X.iloc[train_i], X.iloc[test_i]
y_train, y_test = y.iloc[train_i], y.iloc[test_i]
得到同樣的錯誤:
KeyError: "None of [Int64Index([26], dtype='int64')] are in the [index]"
通過將數據框保存到本地文件並打開它來解決,
如下所示:
df.to_csv('Step1.csv',index=False)
df = pd.read_csv('Step1.csv')
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.