How do I confirm entities are saved with GAE's Eventual Consistency?

Question

I'm trying to create tests to verify that my entities are being saved in the database. When I put breakpoints in the post function, I can see that the customer count changes after the record is saved. I read https://cloud.google.com/appengine/docs/python/tools/localunittesting#Python_Writing_High_Replication_Datastore_tests

From what I understood, the tests were failing because of Eventual Consistency and the way to get around that was to change the PseudoRandomHRConsistencyPolicy settings.

policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)  

And when I ran the test again I got the same error.

What am I doing wrong with creating these tests?

> /Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py(137)post()  
-> customer.put()  
(Pdb) l  
134             query = Customer.query()  
135             orig_customer_count = query.count()  
136             import pdb; pdb.set_trace()  
137  ->         customer.put()  
138             import pdb; pdb.set_trace()  
139             query_params = {'leadbook_name': leadbook_name}  
140             self.redirect('/?' + urllib.urlencode(query_params))  
141       
142     config = {}  
(Pdb) orig_customer_count  
5  
(Pdb) c  
> /Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py(139)post()  
-> query_params = {'leadbook_name': leadbook_name}  
(Pdb) l  
134             query = Customer.query()  
135             orig_customer_count = query.count()  
136             import pdb; pdb.set_trace()  
137             customer.put()  
138             import pdb; pdb.set_trace()  
139  ->         query_params = {'leadbook_name': leadbook_name}  
140             self.redirect('/?' + urllib.urlencode(query_params))  
141       
142     config = {}  
143     config['webapp2_extras.sessions'] = {  
144         'secret_key': 'my-super-secret-key',  
(Pdb) query.count()  
6  

The entities also show up in the Datastore Viewer.

However, my test keeps failing.

$ nosetests --with-gae  
F  
======================================================================  
FAIL: test_guest_can_submit_contact_info (dermalfillersecrets.functional_tests.NewVisitorTest)  
----------------------------------------------------------------------  
Traceback (most recent call last):  
  File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/functional_tests.py", line 80, in test_guest_can_submit_contact_info  
    self.assertNotEqual(orig_custs, query.count())  
AssertionError: 0 == 0   

This is the functional_tests.py file contents:

import os, sys  
sys.path.append("/usr/local/google_appengine")  
sys.path.append("/usr/local/google_appengine/lib/yaml/lib")  
sys.path.append("/usr/local/google_appengine/lib/webapp2-2.5.2")  
sys.path.append("/usr/local/google_appengine/lib/django-1.5")  
sys.path.append("/usr/local/google_appengine/lib/cherrypy")  
sys.path.append("/usr/local/google_appengine/lib/concurrent")  
sys.path.append("/usr/local/google_appengine/lib/docker")  
sys.path.append("/usr/local/google_appengine/lib/requests")  
sys.path.append("/usr/local/google_appengine/lib/websocket")  
sys.path.append("/usr/local/google_appengine/lib/fancy_urllib")  
sys.path.append("/usr/local/google_appengine/lib/antlr3")  

import unittest  
from selenium import webdriver  
from google.appengine.api import memcache  
from google.appengine.ext import db  
from google.appengine.ext import testbed  
import dev_appserver    
from google.appengine.tools.devappserver2 import devappserver2  


class NewVisitorTest(unittest.TestCase):  

    def setUp(self):  
        self.testbed = testbed.Testbed()  
        self.testbed.activate()  
        #self.testbed.setup_env(app_id='dermalfillersecrets')  
        self.testbed.init_user_stub()  
        ####################################################
        # this sets testbed to imitate strong consistency 
        from google.appengine.datastore import datastore_stub_util
        policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)
        self.testbed.init_datastore_v3_stub(consistency_policy=policy)
        self.testbed.init_memcache_stub() 
        ####################################################

        # setup the dev_appserver  
        APP_CONFIGS = ['app.yaml']  

        self.browser = webdriver.Firefox()  
        self.browser.implicitly_wait(3)  

    def tearDown(self):  
        self.browser.quit()  
        self.testbed.deactivate()  

    def test_guest_can_submit_contact_info(self):  
        from main import Customer  
        query = Customer.query()  
        orig_custs = query.count()  
        self.browser.get('http://localhost:8080')  
        self.browser.find_element_by_name('id_name').send_keys("Kallie Wheelock")  
        self.browser.find_element_by_name('id_street').send_keys("123 main st")  
        self.browser.find_element_by_name('id_phone').send_keys('(404)555-1212')  
        self.browser.find_element_by_name('id_zip').send_keys("30306")  
        self.browser.find_element_by_name('submit').submit()  
        # this should return 1 more record  
        #import pdb; pdb.set_trace()  
        query = Customer.query()   
        self.assertNotEqual(orig_custs, query.count())  
        assert(Customer.query(Customer.name == "Kallie Wheelock").get())  
        # Delete the Customer record  
        Customer.query(Customer.name =="Kallie Wheelock").delete()  

Answer 1

The PseudoRandomHRConsistencyPolicy is not helping you here because your selenium test is submitting a live html form and the subsequent db update happening on the server which is outside scope of your policy.

What you testing here is the end to end testing not the unit test per se. So your selenium test should take care of the real world scenario and should wait for a predefined period of time before comparing the counts.

Answer 2

There's nothing wrong with strong/eventual consistency, but the design of your tests is wrong. Why you're trying to deal with devappserver in your tests by yourself? Why you're trying to remove entities in the end of the test? Each test should be isolated from each other and start from empty datastore with some possible initializations.

Please, use the latest version of NoseGAE plugin. Here's two simple tests about strong/eventual consistency:

import unittest
from google.appengine.ext import ndb
from google.appengine.datastore import datastore_stub_util


class Foo(ndb.Model):
  pass


class TestEventualConsistency(unittest.TestCase):
  nosegae_datastore_v3 = True
  nosegae_datastore_v3_kwargs = {
    'consistency_policy': datastore_stub_util.PseudoRandomHRConsistencyPolicy(
      probability=0)}

  def test_eventual_consistency(self):
    self.assertEqual(Foo.query().count(), 0)
    Foo().put()
    self.assertEqual(Foo.query().count(), 0)


class TestStrongConsistency(unittest.TestCase):
  nosegae_datastore_v3 = True
  nosegae_datastore_v3_kwargs = {
    'consistency_policy': datastore_stub_util.PseudoRandomHRConsistencyPolicy(
      probability=1)}

  def test_strong_consistency(self):
    self.assertEqual(Foo.query().count(), 0)
    Foo().put()
    self.assertEqual(Foo.query().count(), 1)

Notice that I don't have anything about GAE paths, dev_appserver, etc. You still can control testbed by yourself, but better configure it with nosegae_*. (read about this in plugin documentation)

And as I remember, it will work even if you will programmatically fill your HTML form, but its not unittests anymore though.

Answer 3

Try using ancestor queries to get strong consistency instead of eventual consistency. From the docs:

Ancestor queries allow you to make strongly consistent queries to the datastore...

If this does not work, the next thing I would try is to not reuse the query object but create a new the second time.

If this does not work either, my guess is that something else is wrong. I am not familiar with browser test, but I have used webtest with great success for testing web endpoints and have not had any consistency issues while unit testing.

Answer 4

Queries are eventually consistent (unless an ancestor is set), but a get operation is always consistent.

If your objective is to simply test the code for writing an entity, you can insert an entity in this test and check if you can retrieve this entity using its key.