Seeking Common Words/Phrases in ActiveRecord

Question

Rails with PG in production but SQLite3 on development. App is a survey with several questions. I have a Model for each question response. I have a query that pulls the responses for each question individually.

I want to be able to pick out common words and/or phrases from that ActiveRecord result to build something like a "word cloud". Is there a gem that would be useful to do this? Otherwise is there a code example/tutorial that runs through this. I'm not certain of what to search for to find an answer, but searching for "word cloud" just seems to pop up front end design hacks. I'm more concerned with getting the data.

Answer 1

You can offload the word counting logic to PSQL:
This query should return each word in some_column for some_table alongside its number of occurrences.

SELECT word, count(*) AS word_count
FROM ( 
  SELECT regexp_split_to_table(some_column, '\s') as word
  FROM some_table
) t
GROUP BY word

This can be executed via ActiveRecord like this:

result = ActiveRecord::Base.connection.exec_query(query)

You can combine that with magic_cloud gem( https://github.com/zverok/magic_cloud ) to generate the words cloud as an image, something like this:

words = result.map {|k, v| [v['word'], v['word_count'] ]} #untested
cloud = MagicCloud::Cloud.new(words, rotate: :free, scale: :log)

Or alternatively, return the words as JSON and process it with whichever JS visualization library you like.