使用准备好的数据进行Sci-kit分类

Question

I am trying to use the Sci-kit learn python library to classify a bunch of urls for the presence of certain keywords matching a user profile. A user has name, email address ... and a url assigned to them. I have created a txt with the result of each profile data match on each link so it is in the format:

Name  Email  Address
  0     1      0      =>Relavent
  1     1      0      =>Relavent
  0     1      1      =>Relavent
  0     0      0      =>Not Relavent

Where the 0 or 1 signifies that the attribute was found on the page(each row is a webpage) How do i give this data to the sci-kit so it can use it to run a classifier? The examples i have seen all have data coming from a predefined sch-kit library such as digits or iris or are being generated in the format i already have. I just dont know how to use the data format i have to provide to the library

The above is a toy example and i have many more features than 3

Answer 1

The data needed is a numpy array (in this case a "matrix") with the shape (n_samples, n_features) .

A simple way to read the csv-file to the right format by using numpy.genfromtxt . Also refer this thread .

Let the contents of a csv file (say file.csv in the current working directory) be:

a,b,c,target
1,1,1,0
1,0,1,0
1,1,0,1
0,0,1,1
0,1,1,0

To load it we do

data = np.genfromtxt('file.csv', skip_header=True)

The skip_header is set to True , to prevent reading the header column (The a,b,c,target line). Refer numpy's documentation for more details.

Once you load the data, you need to do some pre-processing based on your input data format. The preprocessing could be something like splitting the input and the targets (classification) or splitting the whole dataset into a training and validation set (for cross-validation).

To split the input (feature matrix) from the output (target vector) we do

features = data[:, :3]
targets = data[:, 3]   # The last column is identified as the target

For the above given CSV data, the arrays will use will look like:

features = array([[ 0, 1, 0],
              [ 1, 1, 0],
              [ 0, 1, 1],
              [ 0, 0, 0]])  # shape = ( 4, 3)

targets = array([ 1, 1, 1, 0])  # shape = ( 4, )

Now these matrices are passed to the estimator objects fit function. If you are using the popular svm classifier then

>>> from sklearn.svm import LinearSVC
>>> linear_svc_model = LinearSVC()
>>> linear_svc_model.fit(X=features, y=targets)