sklearn SVD fit_transform函数的输入数据类型

Question

I have already processed document data in CSV file, which I read in pandas DataFrame:

+----------+------+------------+
| document | term | count      |
+----------+------+------------+
| 1        | 126  | 1          |
| 1        | 80   | 1          |
| 1        | 1221 | 2          |
| 2        | 2332 | 1          |

So it consists of document_id, term, and term frequency.

I don't have original documents, but just this processed data, and I want to apply SVD with sklearn, but I can not figure how to prepare this DataFrame for SVD fit_transform() , which expects:

X : {array-like, sparse matrix}, shape (n_samples, n_features)

Answer 1

You can convert this CSV to libsvm format:

<label> <index1>:<value1> <index2>:<value2> ...
.
.
.

So, your example data will look like:

0 80:1 126:1 1221:2
0 2332:1

Then read this file using sklearn.datasets.load_svmlight_file

from sklearn.datasets import load_svmlight_file
X, y = load_svmlight_file('your_libsvm_format_file.libsvm')

then,

from sklearn.decomposition import SVD
svd = SVD()
X_transformed = svd.fit_transform(X)