如何获取SKLearn的ndarray格式的数据？

Question

Scikit-Learn is a great Python module that supplies a support vector machine with many algorithms. I've been learning how to use the module for the past few days, and I've noticed it relies heavily on the separate numpy module.

I understand what the module does, but I'm still learning about how it works. Here is a very brief example of what I'm using sklearn for:

from sklearn import datasets, svm
import numpy

digits = datasets.load_digits() #image pixel data of digits 0-9 as well as a chart of the corresponding digit to each image

clf = svm.SVC(gamma=0.001,C=100) #SVC is the algorithm used for classifying this type of data

x,y = digits.data[:-1], digits.target[:-1] #feed it all the data
clf.fit(x,y) #"train" the SVM

print(clf.predict(digits.data[0])) #>>>[0]
#with 99% accuracy, all of the data consists of 1797 samples.
#if this number gets smaller, the accuracy decreases. with 10 samples (0-9),
#accuracy can still be up to as high as 90%.

That's very basic classification.

Using the following code with matplotlib.pyplot:

import matplotlib.pyplot as plt #in shell after running previous code
plt.imshow(digits.images[0],cmap=plt.cm.gray_r,interpolation="nearest")
plt.show()

gives the following image:

The first pixel (left to right, top to bottom, like reading) would be represented by a 0. Same for the second, but the third would be represented by 7 or something (range is 0 to 15), fourth being about 13. Here's the actual data for the image:

[[  0.   0.   5.  13.   9.   1.   0.   0.]
 [  0.   0.  13.  15.  10.  15.   5.   0.]
 [  0.   3.  15.   2.   0.  11.   8.   0.]
 [  0.   4.  12.   0.   0.   8.   8.   0.]
 [  0.   5.   8.   0.   0.   9.   8.   0.]
 [  0.   4.  11.   0.   1.  12.   7.   0.]
 [  0.   2.  14.   5.  10.  12.   0.   0.]
 [  0.   0.   6.  13.  10.   0.   0.   0.]]

if I wanted to classify text data, for example forum posts in the wrong subforum/category, how would I convert that data into the number system used in this dataset example?

Answer 1

For each sample(eg each Forum Post) you must have a vector(in python a list). For example if you have 200 post and their respective category, you must have 200 list for training data and exactly one list that has 200 element for each 200 category. each list of traning category can be a model(eg Bag Of Word. see here: https://en.wikipedia.org/wiki/Bag-of-words_model ). Notice that all list for training must have same element(same dimention)(for example each list must have 3000 element that each element reperesnt present or absent of a word) Try to look at this, it's easy for begginers: https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-1-for-beginners-bag-of-words