will_paginate in rails when using join and group by

Question

I noticed that when using will_paginate 20 results a page, joining multiple tables group_by "nickname", the output is paginated but 3 "nicknames" only showing (this makes sense since pagination counted the output before group by) but how can I solve that? also, I want to display the output like that and limit the number of items per page based on "nickname" column: please note that the "data" table has many "preferences". (data.id = preferences.data_id)

{
"totalCount": 123,
  "pageInfo": {
    "currentPage": 1,
    "nextPage": 2,
    "lastPage": 8
  },
  "results": [
    {
      "data": {
        "id": 1,
        "nickname": "foo"
      },
      "preferences": [
        {
          "id": 4479,
          "created_at": "2019-05-21T00:39:45.772Z",
          "updated_at": "2019-05-21T00:39:45.772Z",
          "title": "Check Database",
          ...
        },
        ...
      ]
    },
    ...
  ]
}

  data_res = Data.paginate(page: params[:page], per_page: 
  20).joins("INNER JOIN preferences ON data.id = 
  preferences.data_id").select("data.*, preferences.*").order(id: :asc)
  data_group_by = data_res.group_by { |r| r.nickname }
  respond_to do |format|
        format.any
        format.json {
          render :json => {
              :totalCount => data_res.total_entries,
              :pageInfo => {
                  :currentPage => data_res.current_page,
                  :nextPage => data_res.next_page,
                  :total_pages => data_res.total_pages,
                  :per_page => data_res.per_page,
              },
              :results =>  data_res
          }
        }
      end

Answer 1

If I'm understanding your question correctly(probably not), pagination says it has 20 records, but you're only seeing 3 records being returned because they're grouped?

However, what you want is 20 records, with 20x preferences grouped?

If that's the case, I think you're probably overcomplicating your query.

1. I don't think you should use the select("data.*, preferences.*") because it basically just adds a new record per preference fetched, so the preferences is probably the determinant of how many records you're getting rather than data on which you're paginating on + you're dynamically adding additional methods to each of the data returned to account for the preferences
2. data_res.group_by { |r| r.nickname } data_res.group_by { |r| r.nickname } seems unnecessary, unless you have data records that are not unique, in which case I'd question the reason for grouping them by that.

In my opinion, if nicknames are unique, ie there can only be 1 data record with the same nickname , here's what I'd propose

class Data
   has_many :preferences
end
class Preference
   belongs_to :data
end

joins-ing and includes-ing here to ensure the preferences are eager loaded, while conforming to your existing query of only fetching data with preferences

data_res = Data.joins(:preference).includes(:preference).paginate(page: params[:page], per_page: 20).order(id: :asc) # this should give you 20 records of data that has preferences

Then your serializer can do the rest of work to ensure your data is properly mapped and that your preferences are on the same level(there are several ways to achieve that with any serialization package), eg

class DataPreferencesSerializer < AMS
  attributes :data
             :preferences # or can be has_many :preferences serializer
  def data
   {id: object.id, nickname: object.nickname }
  end

  def preferences
   object.preferences
  end
end

results= ArraySerializer.new(data_res, each_serializer: DataPreferencesSerializer, root: false)

Like I said, there are several ways of achieving the serialization, so the implementation above is just an idea of the approach you might take and not a specific implementation.

PS: Your INNER JOIN ensures that all the returned data records have an associated preferences , so any data that doesn't have at least one preference that is probably excluded from the records you get back.