使用dplyr删除停用词

Question

Reading http://tidytextmining.com/tidytext.html states :

"

Often in text analysis, we will want to remove stop words; stop words are words that are not useful for an analysis, typically extremely common words such as “the”, “of”, “to”, and so forth in English. We can remove stop words (kept in the tidytext dataset stop_words) with an anti_join().

data(stop_words)

tidy_books <- tidy_books %>% anti_join(stop_words)

"

I'm attempting to modify to remove stop words from a string :

data(stop_words)
str_v <- paste(c("this is a test"))
str_v <- str_v %>%
  anti_join(stop_words)

but returns error :

Error in UseMethod("anti_join") : 
  no applicable method for 'anti_join' applied to an object of class "character"

Do need to convert str_v to class that contains method anti_join ?

Answer 1

The str_v is a vector. It needs to be converted to a data.frame or tibble using as.tibble , then with unnest_tokens the 'value' column is split up into words while renaming it as 'word', so that when we do the anti_join the common columns match up and join by 'word'

library(tidytext)
library(tibble)
library(dplyr)
str_v %>%
    as.tibble %>% 
    unnest_tokens(word, value) %>%
    anti_join(stop_words)
# A tibble: 1 x 1
#   word
#  <chr>
#1  test