fit_transform之后的数组大小不同

Question

I have a problem with fit_transform function. Can someone explain why size of array different?

In [5]: X.shape, test.shape

Out[5]: ((1000, 1932), (1000, 1932))

In [6]: from sklearn.feature_selection import VarianceThreshold
        sel = VarianceThreshold(threshold=(.8 * (1 - .8)))
        features = sel.fit_transform(X)
        features_test = sel.fit_transform(test)

In [7]: features.shape, features_test.shape

Out[7]:((1000, 1663), (1000, 1665))

UPD: Which transformation can help me get arrays with same sizes?

Answer 1

It is because you are fitting your selector twice .

First, note that fit_transform is just a call to fit followed by a call to transform .

The fit method allows your VarianceThreshold selector to find the features it wants to keep in the dataset based on the parameters you gave it.

The transform method performs the actual feature selection and returns an array with just the selected features.

Answer 2

Because fit_transform applies a dimensionality reduction on the array. This is why the resulting arrays dimensions are not the same as the input.

See this what is the difference between 'transform' and 'fit_transform' in sklearn and this http://scikit-learn.org/stable/modules/feature_extraction.html