R文本挖掘问题

Question

I am totally new to programming and I am now doing my ResMA where I have started learning R.

I have to do something very very simple now and I seem to fail at some point. I just have to count the graphemes (the letters) in one txt file, nothing else. I am first creating a corpus with TM, I am cleaning it and everything, but when I try to run the frequency analysis of each grapheme, the text is actually not cleaned of punctuation and strange symbols, etc.

The code I am using is this:

library("tm")
library("SnowballC")
library("wordcloud")
library("RColorBrewer")
filePath <- choose.files()
text <- readLines(filePath)
docs <- Corpus(VectorSource(text))
toSpace <- content_transformer(function (x , pattern ) gsub(pattern, " ", x))
docs <- tm_map(docs, toSpace, "/")
docs <- tm_map(docs, toSpace, "@")
docs <- tm_map(docs, toSpace, "\\|")
docs <- tm_map(docs, content_transformer(tolower))
docs <- tm_map(docs, removeNumbers)
docs <- tm_map(docs, removePunctuation)
docs <- tm_map(docs, stripWhitespace)
docs <- tm_map(docs, stemDocument)
print (docs)

x=strsplit(text, "")

newlist = unlist(x,recursive=F)
freqtab = table(newlist)

print (freqtab)

Ok, so, it is obvious that docs here is totally different than the x down there, but when I try to do the things vice-versa, it is still not working.

I just need to do this: I am going to school!---> i am going to school----> i- 2 a- 1 m- 1 ....

I don't get where my problem is coming from, I will appreciate your help!

Answer 1

The problem is that you are not modifying text with all your operations; you are working over docs .

Running your code using the simple example in your post as text ,

text <- "I am going to school!"
docs <- Corpus(VectorSource(text))
toSpace <- content_transformer(function (x , pattern ) gsub(pattern, " ", x))
docs <- tm_map(docs, toSpace, "/")
docs <- tm_map(docs, toSpace, "@")
docs <- tm_map(docs, toSpace, "\\|")
docs <- tm_map(docs, content_transformer(tolower))
docs <- tm_map(docs, removeNumbers)
docs <- tm_map(docs, removePunctuation)
docs <- tm_map(docs, stripWhitespace)
docs <- tm_map(docs, stemDocument)

and then printing the content of docs , all your modifications apply

print(unlist(docs)[1])
    content.content 
"i am go to school" 

although note that because of the stemmer, "going" is transformed to "go".

Then you can count the characters as in your original code,

x=strsplit(as.character(unlist(docs)[1]), "")
freqtab = table(x[[1]])
print(freqtab)

  a c g h i l m o s t 
4 1 1 1 1 1 1 1 4 1 1

Hope it helps.