R - weird error/warning after SVM training (e1071)

Question

I get a strange error after training the e1071 SVM. It is a text-document multiclass classification, on a large (10000x1000) sparse matrix (DTM). It seems that something is wrong with the features (columns).

The summary(svmModel) works. The results could be better (as always (; ).

However, something is wrong and this may be a reason why results are inconsistent.

> svmModel <- svm(labels ~., data= train[,-1], cross = 10, seed = 1234, kernel="linear")

Warning message:
In svm.default(x, y, scale = scale, ..., na.action = na.action) :
  Variable(s) ‘abgebildet’ and 
...
‘could’ and  [... truncated]

Answer 1

Check in your training dataset for variables with no values. One way to do this is by taking sum of all the columns.

colSums(train[,!colnames(train)=yvar])

If the value is 0 for an independent variable that I can't remove, I usually take a stratified sample as the training dataset. It is usually done for a flag variable taking values 0 and 1.

#stratified sampling
library(sampling)
Training<- strata(train, stratanames = "emptyvar", size = c(1000,500))
#this creates a sample of size 1000 and 500 for 0 and 1 each
strata.train<-getdata(train,Training)
#it creates additional 3 columns which you can remove
train<-strata.train[,!colnames(strata.train) %in% c("ID_unit","Prob","Stratum")]

On the other hand you can also add, scale=F to your svm() and scale your variables beforehand. This avoids the svm function from scaling your variables which leads to z value being an NaN where variables are empty. However, you'd want to scale your variables which you can do manually.

cols<-c(1:5) #say you want to scale the first 5 variables
library(plyr)
standardize <- function(x) as.numeric((x - mean(x)) / sd(x))
train[cols] <- plyr::colwise(standardize)(train[cols])

Answer 2

If there are words which occur rarely then it is not unlikely that the corresponding features in the training data might have only 0's. I believe that this can cause this warning.