google app engine cross group transactions needing parent ancestor

Question

From my understanding, @db.transactional(xg=True) allows for transactions across groups, however the following code returns "queries inside transactions must have ancestors".

@db.transactional(xg=True)
def insertUserID(self,userName):
    user = User.gql("WHERE userName = :1", userName).get()
    highestUser = User.all().order('-userID').get()
    nextUserID = highestID + 1 
    user.userID = nextUserID
    user.put()

Do you need to pass in the key for each entity despite being a cross group transaction? Can you please help modify this example accordingly?

Answer 1

An XG transaction can be applied across max 25 entity groups. Ancestor query limits the query to a single entity group, and you would be able to do queries within those 25 entity groups in a single XG transaction.

A transactional query without parent would potentially include all entity groups in the application and lock everything up, so you get an error message instead.

In app engine one usually tries to avoid monotonically increasing ids. The auto assigned ones might go like 101, 10001, 10002 and so on. If you know that you need monotonically increasng ids it and it'll work for you performance wise, how about:

1. Have some kind of model representation of userId to enable key_name usage and direct lookup

2. Query for userId outside transaction, get highest candidate id

3. In transaction do get_or_insert; lookup UserId.get_by_key_name(candidateid+1). If already present and pointing to a different user, try again with +2 and so on until you find a free one and create it, updating the userid attribute of user at the same time.

If the XG-transaction of updating UserId+User is too slow, perhaps create UserId+task in transaction (not XG), and let the executing task associate UserId and User afterwards. Or a single backend that can serialize UserId creation and perhaps allow put_async if you retry to avoid holes in the sequence and do something like 50 creations per second.

If it's possible to use userName as key_name you can do direct lookup instead of query and make things faster and cheaper.

Answer 2

Cross group transactions allow you to perform a transaction across multiple groups, but they don't remove the prohibition on queries inside transactions. You need to perform the query outside the transaction, and pass the ID of the entity in (and then check any invariants specified in the query still hold) - or, as Shay suggests, use IDs so you don't have to do a query in the first place.

Answer 3

Every datastore entity has a key, a key (amount other things) has a numeric id that the AppEngine assign to it or key_name which you can give it.

In your case it looks like you can use the numeric id, after you call put() on the user entity you will have: user.key().id() (or user.key.id() if your using NDB) which will be unique for each user (as long as all the user have the same parent, which is None in your code).

This id is not sequential but guarantee to be unique.