Google云数据存储模拟器初始化数据

Question

I would like to use the google cloud datastore emulator for a local project. I have already installed and ran the emulator with :

gcloud beta emulators datastore start

My app connects to it, but the problem is that I do not know how to fill it with entities since there is no user interface and my application requires some admin users to be present.

I also tried to export the production database (datastore) with the following command:

 gcloud datastore export

but couldn't make it work.

Should I write a standalone js/python script that fills the database programatically ?

Please advise

Answer 1

The emulator creates a "Datastore" working on your local machine that basically emulates the behavior as if it was the Google Cloud Datastore itself.

So, if you are already running the emulator and your app connects to it, simply using any script that connects to Datastore you will be able to perform any read/write operation. So for example, if you use this python-datastore github repo :

the code inserts entities of every "user's ip" & "timestamp" when they visited your app, and then queries last 10 visits:

entity = datastore.Entity(key=ds.key('visit'))
entity.update({
    'user_ip': user_ip,
    'timestamp': datetime.datetime.utcnow()
})

ds.put(entity)

query = ds.query(kind='visit', order=('-timestamp',))

results = [
    'Time: {timestamp} Addr: {user_ip}'.format(**x)
    for x in query.fetch(limit=10)]

output = 'Last 10 visits:\n{}'.format('\n'.join(results))

So if you are running your App using the emulator, these all entities will be inserted in local and queried from there. If you stop the emulator and then run it again you will see something like:

Reusing existing data in [/tmp/tmp.(whatever)/emulators/datastore]

so you will be able to keep using the same data unless you erase it or change the emulator's data directory changing the --data-dir flag

If you run the following command:

gcloud datastore export

First of all you are missing the OUTPUT_URL_PREFIX ; where your datastore will be exported. And second, this command doesn't have the functionality to work with local datastore yet:You can see the following public issue tracker where it has been already requested.

There is a way of exporting your Production datastore to your local one as you can see in the answer and edit by @Olivier.Roger and @stanzheng in the following thread . You have to follow these steps:

1.Deploy some App which is runnning using the remote_api . For example this repo is a straightforward way.

2.Run this command to download your datastore in production to the file data.csv:

appcfg.py download_data -A YOUR_APP_NAME --url=http://YOUR_APP_NAME.appspot.com/_ah/remote_api/ --filename=data.csv

3.Start the datastore emulator:

gcloud beta emulators datastore start

4.Run the Local Development Server with the same remote_api repo than before . When you run this you will see something like:

Starting API server at: http://0.0.0.0:39693

Use this last port(39693) in the following step

5.Run the following command:

appcfg.py --url=http://localhost:39693/_ah/remote_api/ --filename=data.csv upload_data

In the last step what you are actually doing is the following: You are uploading the data.csv to your App running in local. Taking into account that you are also running the datastore emulator, your app running in local is connected to it, so you're uploading the data.csv to your local datastore.