[英]training data with GridSearchCV gives me ValueError, Sci-kit learn
[英]sci-kit learn crashing on certain amounts of data
我正在嘗試處理一個numpy數組,其中包含71,000行200列浮點數,當我超過5853行時,我正在嘗試的兩個sci-kit學習模型都會出現不同的錯誤。 我嘗試刪除有問題的行,但它仍然失敗。 sci-kit可以學習不處理這么多數據,還是其他什么? X是列表列表的numpy數組。
KNN:
nbrs = NearestNeighbors(n_neighbors=2, algorithm='ball_tree').fit(X)
錯誤:
File "knn.py", line 48, in <module>
nbrs = NearestNeighbors(n_neighbors=2, algorithm='ball_tree').fit(X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/neighbors/base.py", line 642, in fit
return self._fit(X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/neighbors/base.py", line 180, in _fit
raise ValueError("data type not understood")
ValueError:數據類型未被理解
K-方式:
kmeans_model = KMeans(n_clusters=2, random_state=1).fit(X)
錯誤:
Traceback (most recent call last):
File "knn.py", line 48, in <module>
kmeans_model = KMeans(n_clusters=2, random_state=1).fit(X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/cluster/k_means_.py", line 702, in fit
X = self._check_fit_data(X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/cluster/k_means_.py", line 668, in _check_fit_data
X = atleast2d_or_csr(X, dtype=np.float64)
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 134, in atleast2d_or_csr
"tocsr", force_all_finite)
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 111, in _atleast2d_or_sparse
force_all_finite=force_all_finite)
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 91, in array2d
X_2d = np.asarray(np.atleast_2d(X), dtype=dtype, order=order)
File "/usr/local/lib/python2.7/dist-packages/numpy/core/numeric.py", line 235, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.
請檢查dtype
的矩陣X
,例如,通過鍵入X.dtype
。 如果是object
或dtype('O')
,則將X
行的長度寫入數組:
lengths = [len(line) for line in X]
然后通過調用來查看是否所有行都具有相同的長度
np.unique(lengths)
如果輸出中有多個數字,那么您的線路長度是不同的,例如從線路5853開始,但可能不是所有時間。
Numpy數據數組僅在所有行具有相同長度時才有用(如果沒有,它們將繼續工作,但不會按預期執行)。 您應該檢查是什么導致了這一點,糾正它,然后返回knn
。
以下是行長不相同時會發生什么的示例:
import numpy as np
rng = np.random.RandomState(42)
X = rng.randn(100, 20)
# now remove one element from the 56th line
X = list(X)
X[55] = X[55][:-1]
# turn it back into an ndarray
X = np.array(X)
# check the dtype
print X.dtype # returns dtype('O')
from sklearn.neighbors import NearestNeighbors
nbrs = NearestNeighbors()
nbrs.fit(X) # raises your first error
from sklearn.cluster import KMeans
kmeans = KMeans()
kmeans.fit(X) # raises your second error
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.