PSQL sqlalchemy paginate JSONB array

Question

I have a model which includes JSONB column

class PgQueryResult(SqlalchemyHelper.Base, PGNormModel):
    """
    Query SQLAlchemy model.
    """

    __tablename__ = 'query_results'

    class QueryResultViewType(Enum):
        TABULAR = "tabular"
        CHART = "chart"

    id = Column(Integer, primary_key=True)
    ingest_timestamp = Column(DateTime(timezone=True), nullable=False)
    query_id = Column(UUID(), ForeignKey('queries.id', ondelete="CASCADE"), nullable=False)
    run_number = Column(BigInteger, nullable=False)
    result = Column(MutableDict.as_mutable(JSONB), nullable=False)
    chart_configuration = Column(MutableDict.as_mutable(JSONB))
    view_type = Column(SqlEnum(QueryResultViewType, create_constraint=False, native_enum=False, length=256),
                    default=QueryResultViewType.TABULAR, nullable=False)

result structure is following:

{
   rows: [['1', '2'], ['3', '4'], ['1', '2'], ['3', '4'] ... + 100000  array],
   columns: ['f1', 'f2']
}

I have the following query:

q = cls.query.filter(
    cls.query_id == query_id,
    cls.ingest_timestamp > datetime.now() - timedelta(days=365),
).order_by(
    cls.query_id,
    cls.run_number.desc(),
    cls.ingest_timestamp.desc(),
)

this query returns all rows inside result and I need to include pagination here. How can I do this?

Answer 1

The logic for your pagination in that case should be at the application side, and to db/query you have to pass only start/end indexes of an array.

In postgresql 12 you have jsonb_path_query_array function. You may used this function in sqlalchemy thanks to func procedure and define slice from an array (eg from 0 index to 300).

from sqlalchemy import func

session.query(func.jsonb_path_query_array(PgQueryResult.result['rows'], '$0-300'))

If you have older postgres version you can find some inspiration for slicing here: https://dba.stackexchange.com/questions/163761/returning-a-json-array-slice-in-postgresql