I want to create my own datasets, and use it in scikit-learn. Scikit-learn has some datasets like 'The Boston Housing Dataset' (.csv), user can use it by:
from sklearn import datasets
boston = datasets.load_boston()
and codes below can get the data
and target
of this dataset:
X = boston.data
y = boston.target
The question is how to create my own dataset and can be used in that way? Any answers is appreciated, Thanks!
Here's a quick and dirty way to achieve what you intend:
import numpy as np
import csv
from sklearn.datasets.base import Bunch
def load_my_fancy_dataset():
with open('my_fancy_dataset.csv') as csv_file:
data_file = csv.reader(csv_file)
temp = next(data_file)
n_samples = int(temp[0])
n_features = int(temp[1])
data = np.empty((n_samples, n_features))
target = np.empty((n_samples,), dtype=np.int)
for i, sample in enumerate(data_file):
data[i] = np.asarray(sample[:-1], dtype=np.float64)
target[i] = np.asarray(sample[-1], dtype=np.int)
return Bunch(data=data, target=target)
5,3,first_feat,second_feat,third_feat
5.9,1203,0.69,2
7.2,902,0.52,0
6.3,143,0.44,1
-2.6,291,0.15,1
1.8,486,0.37,0
In [12]: import my_datasets
In [13]: mfd = my_datasets.load_my_fancy_dataset()
In [14]: X = mfd.data
In [15]: y = mfd.target
In [16]: X
Out[16]:
array([[ 5.90000000e+00, 1.20300000e+03, 6.90000000e-01],
[ 7.20000000e+00, 9.02000000e+02, 5.20000000e-01],
[ 6.30000000e+00, 1.43000000e+02, 4.40000000e-01],
[ -2.60000000e+00, 2.91000000e+02, 1.50000000e-01],
[ 1.80000000e+00, 4.86000000e+02, 3.70000000e-01]])
In [17]: y
Out[17]: array([2, 0, 1, 1, 0])
Assuming you have your data loaded into memory as a 2D array, there is an easy way to do this with OneHotEncoder :
from sklearn import svm
from sklearn.preprocessing import OneHotEncoder
data = [['Male', 1, True], ['Female', 3, True], ['Female', 2, False], ['Male', 2, False]]
y = [0, 1, 1, 1] # expected outputs
enc = OneHotEncoder(drop='if_binary') # create encoder obj that drops unneeded columns on binary inputs
X = enc.fit_transform(data).toarray() # vectorize input data
clf = svm.SVC(gamma=0.001, C=100.) # create classifier obj
clf.fit(X[:-1], y[:-1]) # fit model using training data (all but last entry)
out = clf.predict(X[-1:]) # resulting prediction (last entry only)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.