合并多个ndb不等式查询的结果

Question

The NDB Datastore forbids multiple inequality queries on different properties. To get around this, I thought that the solution might be to combine the results of multiple independent queries. I found this 2011 question which recommends geohashing, with which I am not familiar. So, perhaps there is a better solution today.

Consider these two queries:

q1 = User.query(User.age < 18).fetch()
q2 = User.query(User.city != 'New York City').fetch()

I attempt to join them like this:

results = set(q1).intersection(q2)

However, I encounter TypeError: Model is not immutable .

My questions:

• Is there a better way to deal with multiple inequality filters on different properties?
• If not, how can I resolve the TypeError above?

Thank you for the assistance.

Answer 1

If you can restructure your User model, you could put in some more properties to make the queries simpler. For example, if you query on the same age ranges, then make a property that encodes the ranges:

age_range = ndb.IntegerProperty()    # 0 = 0-17, 1 = 18-29, 2 = 30-39, etc.

Then you can have:

q1 = User.query(User.age_range == 0).query(User.city != 'New York City').fetch()

If your data set is small enough, you can use @TimHoffman's approach:

q1 = User.query(User.age < 18).fetch(keys_only=True)
q2 = User.query(User.city != 'New York City').fetch(keys_only=True)
results = ndb.get_multi(set(q1).intersection(q2))

A more heavyweight approach, that will scale up to big data sets, is the MapReduce library. You can put in multiple filters to reduce your data set.

Answer 2

I had a similar problem. My query was:

@classmethod:
def getUnReadMessages(cls, user, date)
    return cls.query(ndb.AND(cls.created <= date,
                             cls.receiver_key == user.key,
                             cls.status != READ))

But appengine didn't let me do it. So I solved changing one inequality to:

@classmethod:
def getUnReadMessages(cls, user, date)
    return cls.query(ndb.AND(cls.created <= date,
                             cls.receiver_key == user.key,
                             ndb.OR(cls.status == SEND,
                                    cls.status == RECEIVED)))

Problem solved! I hope this helps you.

Answer 3

There are two options:

Change your data models

Add more properties or adapt the current ones so you can query the object according to the restrictions of the Datastore. This may imply categorizing continuous variables.

Find a workaround

You can make the most significant query at first and then manually filter the result. Have in mind the following considerations:

• You can use projection to make the query more efficient.
• Make the results iterable (iter).
• Use get_multi for a list of keys.

Your code may look like this:

query_iter = User.query(User.age < 18).iter(projection=[User.city])
query_keys = [u.key() for u in query_iter if u.city != 'New York City']
query = ndb.get_multi(query_keys)

Or

query = [u for u in User.query(User.age < 18).fetch() if u.city != 'New York City']