sqlalchemy query one-to-many relationship with sqlite

Question

i've written the following example code to build a search index for the ACL system i am writing. The query in this example resturns all objects that have any of the given ACLs assigned. But i need a query/filter that returns objects that have all ACLs assigned.

Any help is appreciated.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import backref
from sqlalchemy import create_engine
from sqlalchemy import ForeignKey
from sqlalchemy import Integer
from sqlalchemy import String
from sqlalchemy import Column

_db_uri = "sqlite:////tmp/test.sql"
Base = declarative_base()
engine = create_engine(_db_uri, echo=False)
Session = sessionmaker(bind=engine)

class IndexObject(Base):
    """ Index object. """
    __tablename__ = 'objects'
    id = Column(Integer, primary_key=True)
    name = Column(String(128), unique=True, nullable=True)
    acls = relationship('IndexObjectACL',
                                cascade = "all,delete",
                                backref='objects',
                                lazy='dynamic')

    def __repr__(self):
        _repr_ =("<IndexObject (name='%s')>" % (self.name))
        return _repr_


class IndexObjectACL(Base):
    """ Index object ACL. """
    __tablename__ = 'acls'
    id = Column(Integer, primary_key=True)
    value = Column(String(128), nullable=False)
    oid = Column(Integer, ForeignKey('objects.id'))

    def __repr__(self):
        __repr__ = ("<IndexObjectACL (value='%s')>" % (self.value))
        return __repr__


object_list = [
        "object1",
        "object2",
        "object3",
    ]

acl_list = {
        "object1" : [
                    "view",
                    "edit",
                    "enable",
                    "delete",
                    ],
        "object2" : [
                    "view",
                    "edit",
                    ],
        "object3" : [
                    "enable",
                    "delete",
                    ],
    }


Base.metadata.create_all(engine)

session = Session()

for o in object_list:
    acls = []
    for acl in acl_list[o]:
        a = IndexObjectACL(value=acl)
        acls.append(a)

    index_object = IndexObject(name=o, acls=acls)
    session.add(index_object)

session.commit()


search_acls = [ "enable", "delete" ]
q = session.query(IndexObject)
q = q.join(IndexObject.acls).filter(IndexObjectACL.value.in_(search_acls))

print(q.all())

session.close()

Answer 1

I think this could be an opportunity to use division – in a way. IndexObjectACL divided by SearchAcls should yield IndexObject s that have all the SearchAcls . In other words query for IndexObject s for which no such SearchAcls exist that are not in its IndexObjectACL s:

from sqlalchemy import union, select, literal

# Create an aliased UNION of all the search_acls to query against
search_acls_union = union(*(select([literal(acl).label('acl')])
                            for acl in search_acls)).alias()

# Query for those IndexObjects where...
# No SearchAcl exists where...
# No IndexObjectACL exists where value == acl AND oid == id
q = session.query(IndexObject).\
    filter(~session.query().select_from(search_acls_union).
           filter(~IndexObject.acls.any(value=search_acls_union.c.acl)).
           exists())

The result of this query is

[<IndexObject (name='object1')>, <IndexObject (name='object3')>]

and if you add

    "object4" : [
                "enable",
                ],
    "object5" : [
                "delete",
                ],

to your acl_list (and the object names to object_list ) for proving that partial matches are not returned, it still returns only objects 1 and 3.

Your original "have any" query could also be rewritten to use a semijoin , or EXISTS in SQL speak:

q = session.query(IndexObject).\
    filter(IndexObject.acls.any(
        IndexObjectACL.value.in_(search_acls)))

Answer 2

queries = []
acl_q = q.join(IndexObject.acls)
for acl in search_acls:
    x = acl_q.filter(IndexObjectACL.value == acl)
    queries.append(x)
q = q.intersect(*queries)

I can try to explain it but i am new to sqlalchemy and SQL in general. So i might explain it the wrong way... The join() joins IndexObject and IndexObjectACL tables based on their relationship which results in a new query. This query is used to create a new query for each ACL we want to match using filter() . Finally we use intersect() ( SQL INTERSECT ) to get all IndexObject that appear in all queries. After some testing it seems like this is a fast way to search objects that have all given ACLs assigned. Its also very pythonic IMHO.