How to select items from one table based on HABTM relation in another? Postgres “ALL” does not work,

Question

I am trying to retrieve mangas (comics) that have certain categories. For example in the code below, I am trying to search for Adventure(id=29) and Comedy(id=25) mangas. I am using "ALL" operator because I want BOTH categories be in mangas. (ie return all Manga that have both a category of 25 AND 29 through the relation table, but can also have other categories attached to them)

@search = Manga.find_by_sql("
    SELECT m.*
    FROM mangas m
    JOIN categorizations c ON c.manga_id = m.id AND c.category_id = ALL (array[29,25])
")

Problems? The query is not working as I am expecting (maybe I misunderstand something about ALL operator). I am getting nothing back from the query. So I tried to change it to

JOIN categorizations c ON c.manga_id = m.id AND c.category_id >= ALL (array[29,25])

I get back mangas whose IDs are GREATER than 29. I am not even getting category #29. Is there something I am missing here?

Also the query is... VERY slow. I would appreciate it if someone comes with a query that return back what I want.

I am using Ruby on Rails 4.2 and postgresql Thanks

Update: (posting models relationship)

class Manga < ActiveRecord::Base
  has_many :categorizations, :dependent => :destroy
  has_many :categories, through: :categorizations
end
class Category < ActiveRecord::Base
  has_many :categorizations, :dependent => :destroy
  has_many :mangas, through: :categorizations
end
class Categorization < ActiveRecord::Base
  belongs_to :manga
  belongs_to :category
end

My attempt based on @Beartech answer:

    wheres = categories_array.join(" = ANY (cat_ids) AND ")+" = ANY (cat_ids)"
    @m = Manga.find_by_sql("
    SELECT mangas.*
    FROM
      (SELECT manga_id, cat_ids
       FROM
       (
         SELECT c.manga_id, array_agg(c.category_id) cat_ids
         FROM categorizations c GROUP BY c.manga_id
        )
        AS sub_table1 WHERE #{wheres}
      )
      AS sub_table2
      INNER JOIN mangas ON sub_table2.manga_id = mangas.id
    ")

Answer 1

I'm adding this as a different answer, because I like to have the other one for historic reasons. It gets the job done, but not efficiently, so maybe someone will see where it can be improved. That said...

THE ANSWER IS!!!

It all comes back around to the Postgresql functions ALL is not what you want. You want the "CONTAINS" operator, which is @> . You also need some sort of aggregate function because you want to match each Manga with all of it's categories, select only the ones that contain both 25 and 29.

Here is the sql for that:

SELECT manga.*
FROM
  (SELECT manga_id, cat_ids
   FROM
     (SELECT manga_id, array_agg(category_id) cat_ids
      FROM categorizations GROUP BY manga_id)
       AS sub_table1 WHERE cat_ids @> ARRAY[25,29] )
    AS sub_table2
  INNER JOIN manga
    ON sub_table2.manga_id = manga.id
;

So you are pulling a subquery that grabs all of the matching rows in the join table, puts their category ids into an array, and grouping by the manga id. Now you can join that against the manga table to get the actual manga records

The ruby looks like:

@search = Manga.find_by_sql("SELECT manga.* FROM (SELECT manga_id, cat_ids FROM (SELECT manga_id, array_agg(category_id) cat_ids FROM categorizations GROUP BY manga_id) AS sub_table1 WHERE cat_ids @> ARRAY[25,29] ) AS sub_table2 INNER JOIN manga ON sub_table2.manga_id = manga.id

It's fast and clean, doing it all in the native SQL.

You can interpolate variables into the .find_by_sql() text. This gives you an instant search function since @> is asking if the array of categories contains all of the search terms.

terms = [25,29]
q = %Q(SELECT manga.* FROM (SELECT manga_id, cat_ids FROM (SELECT manga_id, array_agg(category_id) cat_ids FROM categorizations GROUP BY manga_id) AS sub_table1 WHERE cat_ids @> ARRAY#{terms} ) AS sub_table2 INNER JOIN manga ON sub_table2.manga_id = manga.id")
Manga.find_by_sql(q)

Important

I am fairly certain that the above code is in some way insecure. I would assume that you are going to validate the input of the array in some way, ie

terms.all? {|term| term.is_a? Integer} ? terms : terms = []

Answer 2

Third times the charm, right? LOL

OK, totally changing my answer because it seems like this should be SUPER EASY in Rails, but it has stumped the heck out of me...

I am heavily depending on This answer to come up with this. You should put a scope in your Manga model:

class Manga < ActiveRecord::Base
  has_many :categorizations, :dependent => :destroy
  has_many :categories, through: :categorizations

  scope :in_categories, lambda { |*search_categories|
    joins(:categories).where(:categorizations => { :category_id => search_categories } )
    }

end

Then call it like:

@search = Manga.in_categories(25,29).group_by {|manga| ([25,29] & manga.category_ids) == [25,29]}

This iterates through all of the Manga that contain at least ONE or more of the two categories, makes a "set" of the array of [25,29] with the array from the manga.category_ids and checks to see if that set equals your reqeusted set. This weeds out ALL Manga that only have one of the two keys.

@search will now be a hash with two keys true and false :

{true =>  [#<Manga id: 9, name: 'Guardians of...

     .... multiple manga objects that belong to at least 
          the two categories requested but not eliminated if
           they also belong to a third of fourth category ... ]

 false => [ ... any Manga that only belong to ONE of the two
                categories requested ...  ]
     }

Now to get just the unique Mangas that belong to BOTH categories use .uniq:

@search[true].uniq

BOOM!! You have an array of you Manga objects that match BOTH of your categories.

OR

You can simplify it with:

@search = Manga.in_categories(25,29).keep_if {|manga| ([25,29] & manga.category_ids) == [25,29]}
@search.uniq!

I like that a little bit better, it looks cleaner.

AND NOW FOR YOU SQL JUNKIES

@search = Manga.find_by_sql("Select * 
  FROM categorizations
   JOIN manga ON categorizations.manga_id = manga.id 
   WHERE categorizations.cateogry_id IN (25,29)").keep_if {|manga| ([25,29] & manga.category_ids) == [25,29]}

@search.uniq!

* OK OK OK I'll stop after this one. :-) *

Roll it all into the scope in Manga.rb:

scope :in_categories, lambda { |*search_categories|
joins(:categories).where(:categorizations => { :category_id => search_categories } ).uniq!.keep_if {|manga| manga.category_ids.include? search_categories[0] and manga.category_ids.include? search_categories[1]} }

THERE HAS GOT TO BE AN EASIER WAY??? (actually that last one is pretty easy)