Query on ReferenceProperty

Question

Given that I have the following entities in the datastore:

class Owner(db.Model):
  name = db.StringProperty()
  age = db.IntegerProperty()

class Pet(db.Model):
  name = db.StringProperty()
  owner = db.ReferenceProperty(Owner)

and that some owners have no pets, how best to extract all owners who have a pet, ordered by owner age? I'm aware that JOINS are not possible, so it seems likely that there will be two queries.

I have tried to extract all owner keys from Pets and then have done an 'IN' query with the keys but I hit the maximum 30 subqueries limitation.

Answer 1

I would add an additional property or two to the Owner class.

Define a boolean field owns_pets (or similiar) which you set to True when you add a pet, then select all Owners where owns_pets == True , ordered by Age, then fetch the pets for each Owner using the reverse set.

Alternately add a ListProperty pets containing all the keys of the owned Pets. Then query for all owners (again easier with the boolean above) and then db.get(some_owner.pets)

Without either of these you have a couple of less easy ways.

loop through the set of owners in Age order, fetch reverse reference set (in your case pet_set ) skipping owners where pet_set returns nothing.

Other ways include fetching all pets, collecting the keys of the owners (in a set, removing duplicates) and then db.get(list of owner keys), then order them after the fact in code - not as efficient if you have a lot or possibly not doable (memory/time).) If you want to use this path, have a look at nick johnson prefetch reference set code http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine

Really the best bet is start storing redundant data at write time, that makes often used queries less expensive to perform.