How to access Google App Engine's dev server data from the command line

Question

I regularly download my Google App Engine production data locally (using a custom script that basically calls appcfg.py download_data ) and upload it to the development server so that the data in my development environment matches production. You can find the scripts on Launchpad:

• Download script (called with ./extra/manipular-datos.py --download )
• Launcher script (called with ./extra/launcher.py )

This is what the launcher script does:

• Start the dev server (flushing the database)
• Load the previously downloaded models to the local database
• Change the passwords of all normal and admin users so that we can log in with a password of "toto" for any user in the development environment. This is done with a script that accesses the freshly-loaded datastore data directly.

Everything was working fine until GAE 1.5.2. At that point the development server changed to prefix dev~ , so I added --default_partition='' to launch the dev server. However now the password-changing script seems to not access the loaded data, it always returns 0 entities for normal or admin users. I'm wondering how to access the data from the command line, so explicitely not from an url to prevent this from ever happening on the production server. The scripts in ./extra are not uploaded to GAE, they can thus not be executed from there.

To give you an idea (but look at the script for more context), this is what the script used to do (and was working until GAE <= 1.5.1):

args, option_dict = ParseArguments(['', colCasaBasePath])
config, matcher = LoadAppConfig(colCasaBasePath, {})
SetupStubs(config.application, **option_dict)

The the application's models can be accessed just normally:

from src.models import WebUser, ScyllaUser
for tipo in (WebUser, ScyllaUser):
    usuarios = tipo.all()
    # Now len(usuarios) == 0

What has been changed (after GAE 1.5.1) that breaks my GAE data-accessing script? How would you perform such a workflow?

Answer 1

The trick is to add the --default_partition= argument to the ParseArguments() call in the script that accesses the GAE data:

args, option_dict = ParseArguments(['', colCasaBasePath, "--default_partition="])

Note that there is nothing after the = , that's what put me off. I first tried adding "--default_partition=''" , but that didn't have any effect (but also didn't result in errors).

After adding that call the script is again able to loop over all the users that have been loaded in the dev datastore. You can see the changes I had to make on Launchpad .