TransientError: Temporary search service error - Python Google App Engine Search

Question

I'm trying to run a query and sorting by 'updated_at' field using a large number_found_accuracy:

order_options = search.SortOptions(
    expressions=[search.SortExpression(expression='updated_at',
                                   direction=search.SortExpression.DESCENDING)])

query_options = search.QueryOptions(
    limit=50,
    cursor=search.Cursor(),
    sort_options=order_options,
    number_found_accuracy=25000)

index = search.Index('contacts', namespace='default')
query_future = index.search_async(search.Query("", options=query_options))
contacts = query_future.get_result()

When get_result() is called i get the error bellow:

File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in call rv = self.handle_exception(request, response, e) File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in call rv = self.router.dispatch(request, response) File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher return route.handler_adapter(request, response) File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in call return handler.dispatch() File "/base/data/home/apps/p~imobzi-app/20181127t101400.414282042583891084/modules/base_handler.py", line 72, in dispatch super(BaseHandler, self).dispatch() File "/base/alloc/tmpfs/dynamic_ runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch return self.handle_exception(e, self.app.debug) File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch return method(*args, **kwargs) File "/base/data/home/apps/p~imobzi-app/20181127t101400.414282042583891084/main.py", line 132, in get contacts = query_future.get_result() File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/1/google/appengine/api/search/search.py", line 281, in get_result raise _ToSearchError(e) TransientError: Temporary search service error

The error occurs when i use "number_found_accuracy" and "sort_options" in same query when query result is large (this query return more than 50,000 results).

If "number_found_accuracy" or "sort_options" is removed from query_options i get the result normally, but if both is in query_options, the error occurs.

In a normal case i'd remove "number_found_accuracy" from query, but i need to show the result count for the user and sort it by updated_at field. Does anyone know a way to solve this? This occurs only when a deploy project to server, in a local/development environment, everything works as expected.

Answer 1

One reason for this error can be that the length of the query exceeds the limit of 2000 characters as specified in the documentation .

There is also a limitation on sorting that can be worked around by presorting your documents in the index using Document rank as explained in this StackOverflow answer .

Also note that as per the documentation

number_found:
Returns an approximate number of documents matching the query. QueryOptions defining post-processing of the search results. If the QueryOptions.number_found_accuracy parameter were set to 100, then number_found <= 100 is accurate.

Since you have set number_found_accuracy=25000 , any search result with a size greater than 25000 will be an approximate.