稀疏矩阵上的scikit-learn HashingVectorizer

Question

In scikit-learn, how can I run the HashingVectorizer on data already present in a scipy.sparse matrix?

My data is in svmlight format, so I am loading it with sklearn.datasets.load_svmlight_file and get a scipy.sparse matrix to work on.

The TfidfTransformer from scikit-learn can be fed such a sparse matrix to transform it, but how can I give the same sparse matrix to the HashingVectorizer to use it instead?

EDIT: Is there maybe a series of method calls that can be used on the sparse matrix, maybe using the FeatureHasher ?

EDIT 2: After a useful discussion with the user cfh below, the goal I have is to go from input: a sparse count matrix gotten from svmlight data to output: a matrix of token occurrences, such as the HashingVectorizer is giving. How could this be done?

I provided a sample code below and would really appreciate some help on how to do that, thanks in advance:

from sklearn.feature_extraction.text import TfidfTransformer
import numpy as np
from sklearn.feature_extraction.text import HashingVectorizer
from scipy.sparse import csr_matrix

# example data
X_train = np.array([[1., 1.], [2., 3.], [4., 0.]])
print "X_train: \n", X_train
# transform to scipy.sparse.csr.csr_matrix to be consistent with output from load_svmlight_file
X_train_crs = csr_matrix(X_train)
print "X_train_crs: \n", X_train_crs   
# no problem to run TfidfTransformer() on this csr matrix to get a transformed csr matrix
transformer = TfidfTransformer()
tfidf = transformer.fit_transform(X_train)
print "tfidf: \n", tfidf
# How do I use the HashingVectorizer with X_train_crs ?
hv = HashingVectorizer(n_features=2)

Answer 1

Hashing basically combines words randomly into a smaller number of buckets. With an already computed frequency matrix, you can emulate this like so:

n_features = X_train.shape[1]
n_desired_features = n_features / 5
buckets = np.random.random_integers(0, n_desired_features-1, size=n_features)
X_new = np.zeros((X_train.shape[0], n_desired_features), dtype=X_train.dtype)
for i in range(n_features):
    X_new[:,buckets[i]] += X_train[:,i]

Of course you can adjust the n_desired_features as you wish. Just make sure to use the same buckets for the test data as well.

If you need to do the same for a sparse matrix, you can do this:

M = coo_matrix((repeat(1,n_features), (range(n_features), buckets)),
               shape=(n_features,n_desired_features))
X_new = X_train.dot(M)