在tm包中创建TermDocumentMatrix时出错

Question

I am new to the tm package, and have run into an obstacle when trying to apply the TermDocumentMatrix function.

I have used the following code up until the function fails:

myCorpus <- Corpus(VectorSource(posts$message))
myCorpus <- tm_map(myCorpus, content_transformer(tolower))
myCorpus <- tm_map(myCorpus, removePunctuation)
myCorpus <- tm_map(myCorpus, removeNumbers)

removeURL <- function(x) gsub("http[[:alnum:]]*", "", x)

myCorpus <- tm_map(myCorpus, removeURL)

myStopwords <- c(stopwords("english"))
myCorpus <- tm_map(myCorpus, removeWords, myStopwords)

myCorpusCopy <- myCorpus 
myCorpus <- tm_map(myCorpus, stemDocument)

Upon inspection it seems as if the list of documents is what it should be:

> for(i in 1:5) {
+   cat(paste("[[", i, "]] ", sep =""))
+   writeLines(myCorpus[[i]])
+ }
[[1]] syntel recruitment drive   week  freshers  newregistrationlink    passout graduates
qualification   graduatebebtechmcamemtech
syntel registration link  
limited referrals available 
comment  emailids  reference  future job upd
[[2]] dont miss  opportunity   get placed  one   best mnc companies   world ebay freshers  week  january 
qualification   graduate can apply
ebay registration link  
comment  emailids fast beacuse    referrals left
[[3]] recent passouts      eligible  apply  wipro  go   updated link  lastday reference drive jan  apply link  fresher referral
apply link 
go   link  apply asap
[[4]] robertbosch recruitment drive   week  freshers  newregistrationlink    passout graduates
qualification   graduatebebtechmcamemtech
robertbosch registration link  
limited referrals available 
comment  emailids  reference  future job upd
[[5]] mega job openings   year
mphasis recruitment  freshers january 
qualification   btech bsc bca  graduates mca mba  mtech post graduates
mphasis registration link  
comment  emailids  comment box  reference  future job updates   emailbox    

however, after creating a copy of corpus for stem completion, the problem arises.

myCorpus <- tm_map(myCorpus, stemCompletion,
                   dictionary = myCorpusCopy, lazy = TRUE)
> tdm <- TermDocumentMatrix(myCorpus, control = list(wordLengths = c(1, Inf)))
Error in UseMethod("meta", x) : 
  no applicable method for 'meta' applied to an object of class "try-error"
In addition: Warning messages:
1: In mclapply(x$content[i], function(d) tm_reduce(d, x$lazy$maps)) :
  all scheduled cores encountered errors in user code
2: In mclapply(unname(content(x)), termFreq, control) :
  all scheduled cores encountered errors in user code

Any ideas for a workaround?

Answer 1

I think that you have to recall

myCorpus <- Corpus(VectorSource(myCorpus))

before using the TermDocumentMatrix , your final piece of code will be:

myCorpus <- tm_map(myCorpus, stemCompletion, dictionary = myCorpusCopy)
myCorpus <- Corpus(VectorSource(myCorpus))
tdm <- TermDocumentMatrix(myCorpus, control = list(wordLengths = c(1, Inf)))

If until the stemming of the document no error occured, the previous instructions will solve your problem.

Answer 2

Otherwise, you might try first:

myCorpus <- tm_map(myCorpus, PlainTextDocument)

Before you use

myCorpus <- Corpus(VectorSource(myCorpus))