TermDocumentMatrix in R - 唯一の1グラムが作成されました

Question

Rでtmパッケージを開始したばかりで、問題を解決できないようです。私のトークナイザ機能が正しく動作するように見えるにもかかわらず ：

uniTokenizer <- function(x) NGramTokenizer(x, Weka_control(min=1, max=1)) 
biTokenizer <- function(x) NGramTokenizer(x, Weka_control(min=2, max=2)) 
triTokenizer <- function(x) NGramTokenizer(x, Weka_control(min=3, max=3)) 

uniTDM <- TermDocumentMatrix(corpus, control=list(tokenize = uniTokenizer)) 
biTDM <- TermDocumentMatrix(corpus, control=list(tokenize = biTokenizer)) 
triTDM <- TermDocumentMatrix(corpus, control=list(tokenize = triTokenizer)) 

私は...唯一の1グラムが出てくる、biTDMから同じで

findFreqTerms(biTDM, 50) 

[1] "after" "and"  "most" "the"  "were" "years" "love" 
[8] "you"  "all"  "also" "been" "did"  "from" "get"  

を2グラムを引っ張ってみてください、 2グラムの機能がタクトにあるように表示されます。

x <- biTokenizer(corpus) 
head(x) 

[1] "c in"    "in the"   "the years"  
[4] "years thereafter" "thereafter most" "most of"  

Answer 1

私が唯一の問題はここにあるものと想定することができますNGramTokenizerはVCorpusオブジェクトを必要とするのではなくCorpusオブジェクト。 [最小限の再現可能な例]を含む

library(tm) 
library(RWeka) 

# some dummy text 
text <- c("Lorem ipsum dolor sit amet, consetetur sadipscing elitr", 
      "sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat", 
      "sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum", 
      "Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet") 

# create a VCorpus 
corpus <- VCorpus(VectorSource(text)) 


biTokenizer <- function(x) NGramTokenizer(x, Weka_control(min=2, max=2)) 


biTDM <- TermDocumentMatrix(corpus, control=list(tokenize = biTokenizer)) 

print(biTDM$dimnames$Terms) 

[1] "accusam et"   "aliquyam erat"   "amet consetetur"  "at vero"    "clita kasd"   "consetetur sadipscing" "diam nonumy"   "diam voluptua"   "dolor sit"    "dolore magna"   
[11] "dolores et"   "duo dolores"   "ea rebum"    "eirmod tempor"   "eos et"    "est lorem"    "et accusam"   "et dolore"    "et ea"     "et justo"    
[21] "gubergren no"   "invidunt ut"   "ipsum dolor"   "justo duo"    "kasd gubergren"  "labore et"    "lorem ipsum"   "magna aliquyam"  "no sea"    "nonumy eirmod"   
[31] "sadipscing elitr"  "sanctus est"   "sea takimata"   "sed diam"    "sit amet"    "stet clita"   "takimata sanctus"  "tempor invidunt"  "ut labore"    "vero eos"    
[41] "voluptua at"